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I.       INTRODUCTION 

A.         BACKGROUND 

The  Department  of  Defense  (DOD)  has  experienced  a  decade  of  budget  reductions 
and  downsizing  initiatives  with  additional  cuts  projected  for  the  future.  Increased 
competition  for  shrinking  budget  dollars  and  financial  challenges  associated  with 
downsizing  compel  DOD  organizations  to  use  scarce  resource  dollars  more  efficiently. 
Managers  of  defense  activities  are  specifically  challenged  to  improve  utilization  and 
productivity  rates  in  order  to  ensure  mission  requirements  will  be  met  in  an  environment 
of  reduced  resource  availability. 

The  recent  trend  of  downsizing  and  reduced  budgets  has  had  a  significant  impact 
on  Research,  Development,  Test  and  Evaluation  (RDT&E)  activities  within  the 
Department  of  the  Navy  (DoN).  Functional  realignment,  consolidation,  and 
reorganization  of  Navy  RDT&E  activities  have  been  implemented  in  response  to 
downsizing  initiatives  (Collier,  1998).  There  has  also  been  a  conscious  shift  away  from 
traditional  DOD  resource  allocation  decision  metrics  towards  performance  measures  and 
sound  business  practices  used  by  industry  in  the  private  sector.  Evidence  of  this  new 
direction  is  found  in  the  Defense  Reform  Initiative,  The  Business  Strategy  for  Defense  for 
the  21st  Century,  released  November  10,  1997.  In  this  document,  William  S.  Cohen, 
United  States  Secretary  of  Defense,  states: 


DOD  has  labored  under  support  systems  and  business  practices  that  are  at 
least  a  generation  out  of  step  with  modern  corporate  America.  DOD 
support  systems  and  practices  that  were  once  state-of-the-art  are  now 
antiquated  compared  with  the  systems  and  practices  in  place  in  the 
corporate  world,  while  other  systems  were  developed  in  their  own 
defense-unique  culture  and  have  never  corresponded  with  the  best 
business  practices  of  the  private  sector.  This  cannot  and  will  not  continue. 
(Cohen,  1997) 


To  further  define  best  practices  reform  for  DOD,  a  series  of  initiatives  have  been 
established  in  the  following  major  areas: 

•  Reengineering:    Adopt    modern    business    practices    to    achieve    world-class 
standards  of  performance. 

•  Consolidation:  Streamline  organizations  to  remove  redundancy  and  maximize 
synergy. 

•  Competition:  Apply  market  mechanisms  to  improve  quality,  reduce  costs,  and 
respond  to  customer  needs. 

•  Elimination:  Reduce  excess  support  structures  to  free  resources  and  focus  on 
core  competencies.  (Cohen,  1997) 

This  new  orientation  represents  a  fundamental  change  in  the  way  RDT&E 
activities  evaluate  resource  allocation  alternatives.  As  a  result,  requests  for  RDT&E 
budget  dollars  throughout  the  defense  budgeting  process  will  be  reviewed  with  increased 
emphasis  on  sound  business  justification.  These  new  requirements  significantly  impact 
DON  RDT&E  resource  management  practices.  DON  RDT&E  resource  managers  do  not 
presently  have  an  adequate  system  for  measuring  capacity  utilization  and  productivity  of 
individual  laboratories,  as  evidenced  by  the  following  statement  from  the  Office  of  the 
Under  Secretary  of  Defense,  Acquisition  and  Technology: 


Efforts  to  improve  the  overall  cost  efficiency  of  Defense  laboratories  and 
test  centers  have  been  significantly  limited  by  the  pervasive  absence  of 
accurate,  credible  and  comparable  cost  data.  Current  financial  information 
available  to  RDT&E  management  is  organized  according  to  the  budget 
and  financial  control  process,  a  paradigm  that  emphasizes  level  of  effort 
funding  and  "management  to  budget"  instead  of  cost  control.  In  addition, 
the  limited  cost  data  that  are  available  for  management  review  are  not 
generally  comparable  across  organizations  due  to  the  inconsistent  financial 
methodologies  and  approaches  used  by  the  various  activities  and  services. 
(Memorandum  (i),  1997) 


One  of  the  issues  discussed  in  the  Defense  Reform  Initiative  with  regards  to 
RDT&E  infrastructure  was  that  the  performance  and  cost  of  laboratories  and  test  and 
evaluation  facilities  "can  be  improved  through  a  combination  of  improved  management, 
internal  restructuring,  and  increased  inter-Service  support".  (Memorandum  (ii),  1997) 
These  improvements  can  be  achieved  with  proper  management  of  available  resources, 
increased  efficiency,  and  greater  control  of  costs.  Establishment  of  a  Cost-Based 
Management  Tool  (CBMT)  for  Laboratories  and  Test  and  Evaluation  Centers  has  been 
directed  by  the  Deputy  Secretary  of  Defense  to  provide  executive  level  visibility  of  the 
full  costs  associated  with  laboratories  and  test  and  evaluation  centers.  The  CBMT  is 
structured  so  as  to  provide  accurate  and  credible  information  suitable  for  identifying  cost 
efficiencies  and  financial  best  practices  across  DOD.  (Memorandum  (i),  1997) 

This  executive  level  cost  analysis  tool,  scheduled  to  be  fully  operational  by  the 
end  of  FY  1998,  does  not  provide  for  cost  and  performance  analysis  at  the  individual 
laboratory  level.  DON  RDT&E  activities  are  organized  by  function  and  physically 
operate  as  separate  laboratories.    Each  laboratory  is  designed  to  meet  specific  DOD  and 


DON  requirements  as  part  of  the  overall  RDT&E  mission.  RDT&E  managers,  concerned 
with  improving  efficiency  and  resource  allocation,  need  to  develop  new  strategies  to 
improve  capacity  utilization  and  productivity  of  individual  laboratories.  Proper 
performance  measures  used  to  evaluate  laboratory  capacity  utilization  and  productivity, 
put  into  terms  useful  for  financial  analysis,  will  enable  managers  to  have  better  decision- 
making information. 

The  funding  and  accounting  processes  for  DON  RDT&E  activities  have  changed 
over  the  past  few  years,  significantly  impacting  financial  operating  procedures.  Financial 
information  available  to  managers  for  decision-making  under  the  new  system  is  not 
providing  the  appropriate  data  necessary  for  evaluation  of  productivity  and  capacity 
utilization.  The  introduction  of  the  Navy  Working  Capital  Fund  (NWCF)  as  the  primary 
source  of  RDT&E  funding  has  changed  the  way  laboratories  account  for  costs,  revenues, 
and  laboratory  rate  determination.  This  relatively  new  funding  process,  along  with  its 
associated  accounting  procedures,  has  contributed  to  the  difficulty  of  accurate 
measurement  of  laboratory  performance  in  financial  terms. 

The  new  emphasis  on  efficiency  and  sound  business  practices  from  a  financial 
management  perspective  mandates  that  the  Navy  evaluate  and  incorporate  appropriate 
performance  measurement  tools  for  RDT&E  laboratories.  Laboratory  capacity  utilization 
and  productivity  are  primary  indicators  of  performance.  If  these  tools  are  integrated  into 
the  analysis  process  they  will  increase  the  quality  of  decision-making  information 
available  to  the  resource  manager.    This  thesis  will  help  identify  appropriate  laboratory 


capacity  utilization  and  productivity  measures  and  analyze  the  potential  cost  benefit  of 
their  use. 

B.         OBJECTIVE 

The  primary  objective  of  this  thesis  is  to  identify  capacity  utilization  and 
productivity  measures  for  RDT&E  facilities  to  improve  management  of  DON  resources. 
In  order  to  make  informed  business  decisions  on  spending  and  resource  allocation, 
Research  and  Engineering  managers  must  have  adequate  information  about  the 
productivity  and  cost  efficiency  of  individual  RDT&E  laboratories.  The  performance 
measurements  identified  were  analyzed  using  data  collected  from  NAWCAD  RDT&E 
laboratories  operational  in  FY  1997.  The  results  were  evaluated  for  their  usefulness  as  a 
financial  analysis  and  decision-making  tool.  These  measures  of  laboratory  capacity 
utilization  and  productivity  should  provide  information  necessary  to  improve  the  efficient 
use  of  DON  RDT&E  resources. 


C.         RESEARCH  QUESTIONS 

The  following  questions  were  addressed: 
1.  Primary: 

How  do  we  measure  capacity  utilization  and  productivity  of  DON  RDT&E 

facilities  in  terms  useful  for  financial  and  resource  allocation  decision  making? 


2.  Secondary: 

(1)  What  are  the  plausible  methods  of  measuring  capacity  utilization  and 
productivity  for  RDT&E  facilities? 

(2)  Can  existing  production  capacity  models  be  applied  to  non-production 
environments  such  as  research  laboratories? 

(3)  Are  there  existing  Research  and  Development  benchmark  performance 
measures  in  industry? 

(4)  Can  similar  measures  be  applied  to  DON  RDT&E  laboratory  facilities? 

(5)  Can  dissimilar  laboratories  be  classified  into  categories  useful  for  financial 
performance  comparisons? 

(6)  Can  a  consensus  approach  to  measuring  capacity  utilization  and  productivity 
be  applicable  to  all  RDT&E  activities? 

D.         SCOPE  OF  THESIS 

This  thesis  evaluates  potential  capacity  utilization  and  productivity  measurement 
techniques  and  models  applicable  to  DON  RDT&E  facilities,  incorporating  adjustments 
necessary  to  accurately  capture  the  unique  characteristics  of  research  and  development 
laboratory  activity.  Measures  were  reviewed  for  their  ability  to  provide  quantitative 
analysis  of  laboratory  financial  performance.  The  measurement  methods  identified  as 
appropriate  were  tested  using  data  from  a  sample  of  NAWCAD  RDT&E  laboratories. 
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Results  were  evaluated  for  their  potential  to  improve  the  quality  of  information  available 
for  RDT&E  laboratory  resource  management  and  decision-making. 


E.    METHODOLOGY 

The  methodology  used  for  this  research  was  divided  into  the  following  steps:  (1) 
review  of  the  pertinent  literature  and  existing  models,  (2)  identification  of  laboratory 
classifications  and  categorization,  (3)  review  of  potential  benchmark  measures  existing  in 
industry,  (4)  determination  of  appropriate  performance  measurement  models  and 
techniques,  (5)  collection  of  data,  (6)  application  of  data  in  selected  models,  and  (7) 
analysis  of  results. 

(1)  Literature:  A  review  of  the  literature  on  existing  models  for  capacity 
utilization  and  productivity  was  conducted.  The  findings  were  used  to  select  the  most 
appropriate  performance  measurement  models  and  techniques  applicable  to  the 
NAWCAD  RDT&E  laboratory  environment. 

(2)  Classifications:  Laboratory  organizational  structure,  characteristics  and 
classifications  were  identified  to  establish  a  useful  baseline  of  comparison  among  the 
hundreds  of  different  NAWCAD  RDT&E  laboratories. 

(3)  Industry  Benchmark  Review:  A  review  was  conducted  to  determine  if  existing 
industry  R&D  capacity  utilization  and  productivity  measurement  techniques  could  be 
applied  as  NAWCAD  RDT&E  laboratory  performance  benchmarks. 


(4)  Model:  Appropriate  capacity  utilization  and  productivity  measures  were 
selected.  The  models  considered  were  reviewed  for  their  ability  to  provide  quantitative 
analysis  of  laboratory  performance  while  meeting  NAWCAD  objectives  for  RDT&E 
laboratory  capacity  utilization  and  productivity  management.  Selection  of  the  applicable 
measures  and  model  was  determined  from  a  comparison  of  NAWCAD  RDT&E 
laboratory  management  objectives  to  the  different  model  attributes.  A  model  was  chosen 
that  appeared  to  provide  the  closest  fit  for  the  unique  RDT&E  laboratory  environment. 

(5)  Data:  Data  were  collected  from  a  sample  of  NAWCAD  RDT&E  laboratories. 
An  existing  database  established  for  NAWCAD  Aircrew  Systems  Level  II  Laboratories 
provided  direct  cost,  revenue,  and  square  footage  data  for  the  selected  laboratories. 
Additional  cost  data  were  collected  from  NAWCAD  accounting  records.  Interviews  with 
the  NAWCAD  Comptroller,  laboratory  managers,  and  facility  supervisors,  were 
conducted,  and  a  questionnaire  was  distributed  to  gather  data  about  laboratory  operating 
characteristics  and  equipment  utilization. 

(6)  Model  Application:  The  data  were  used  in  the  selected  model  and  results 
presented  for  analysis  as  measures  of  capacity  utilization  and  productivity  of  RDT&E 
laboratories.  The  sample  of  data  provide  a  baseline  for  laboratory  performance 
measurement  that  may  potentially  apply  to  all  classifications  of  DOD  RDT&E 
laboratories. 

(7)  Analysis:  Analysis  of  research  results  includes  a  review  of  the  model  results 
as  presented  with  NAWCAD  RDT&E  laboratory  data.   The  model  was  evaluated  by  the 


author  for  its  potential  to  improve  the  quality  of  laboratory  performance  information  and 
analysis  provided  for  RDT&E  laboratory  resource  management  and  decision-making. 
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II.   NAWCAD  RDT&E  LABORATORY  ORGANIZATION 


A.    OVERVIEW  OF  NAWCAD  RDT&E  ORGANIZATION 

The  Naval  Air  Warfare  Center  Aircraft  Division  (NAWCAD)  is  a  component  of 
the  Naval  Air  Systems  Command  (NAVA1RSYSCOM)  headquartered  in  Patuxent  River, 
Maryland.  The  primary  mission  of  NAWCAD  is  to  support  research,  development,  test 
and  evaluation  (RDT&E),  engineering  and  fleet  support  of  Navy  and  Marine  Corps  air 
vehicle  systems  and  trainers.  The  full  spectrum  of  the  RDT&E  effort  integrates  a  wide 
range  of  DOD  activities  and  resources.  The  NAWCAD  is  the  steward  of  the  ranges,  test 
facilities,  laboratories,  and  aircraft  necessary  to  support  the  Fleet's  acquisition 
requirements  (Dyer,  1997).  This  thesis  focuses  on  the  laboratory  facilities  and  resources 
located  at  Naval  Air  Station  (NAS)  Patuxent  River,  Maryland,  which  perform  research 
and  development  in  support  of  the  NAWCAD  RDT&E  mission.  Understanding  the 
characteristics  of  NAWCAD  RDT&E  laboratory  operations  and  activities  is  the  first  step 
towards  designing  laboratory  performance  measures  useful  for  business  decision-making. 
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1.  NAWCAD  Laboratory  Facilities  and  Equipment 

There  are  over  five  hundred  laboratories  operated  by  NAWCAD.1  These 
laboratories  are  physically  housed  in  two  primary  buildings  designed  to  accommodate  the 
unique  structural  and  environmental  requirements  of  an  RDT&E  activity.  Each 
laboratory  utilizes  facilities  and  equipment  allocated  to  meet  its  functional  requirements. 
The  specific  space  allocated  to  each  laboratory  can  be  categorized  into  one  of  three  types 
of  space  designs:  (1)  High  Bay  Mechanical,  (2)  Raised  floor  computer  spaces,  and  (3) 
General  purpose  clean  spaces  (Harris,  1998).  A  description  of  each  is  provided  below: 

•  High  Bay  Mechanical  -  Large  square  footage  space  with  two  story  high 
ceiling  to  accommodate  large  equipment  and  test  requirements 

•  Raised  floor  computer  spaces  -  Environmentally   controlled   space   with 
additional  electrical  access  for  computer  and  computer  related  equipment 

•  General  purpose  clean  space  -  standard  electrical  and  environmental  design 
with  minimal  specialized  equipment  design  requirements 

Many  different  types  of  equipment,  from  large  mechanical  devices  to  small 
technical  measurement  tools,  are  owned  and  operated  by  NAWCAD  RDT&E 
laboratories.  They  are  the  tools  that  scientists  and  technicians  use  to  perform  required 
laboratory  RDT&E  activities.  Each  laboratory  houses  specific  types  of  equipment 
designed  to  meet  its  functional  requirements.     The  equipment  types  include  unique. 


1   DOD  defines  laboratory  as  an  activity  (an  aggregate  of  personnel  and  facilities  located 
at  one  base,  under  the  same  command)  owned  and  operated  by  a  DOD  component,  that 
performs  predominantly  science  and  technology,  engineering  development,  systems 
engineering,  engineering  support  of  deployed  material  and  its  modernization,  and/or  in- 
service  engineering  work  (GAO,  1998). 
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highly  specialized  items  such  as  an  ejection  seat  tower  that  simulates  the  actual  aircraft 
ejection  environment,  as  well  as  common,  universal  use  items  such  as  computers  and 
video  recording  devices.  The  laboratories  are  configured  to  accommodate  their 
respective  equipment  requirements,  which  can  be  categorized  into  one  of  the  following 
general  descriptions: 

•  Large  mechanical  devices  -  (equipment/test  items  with  space/utility 
requirements  greater  than  single  floor  space  designs)  e.g.  horizontal 
accelerator  equipment  which  requires  over  300  ft  of  sub-floor  level 
architecture  for  rail/track  design. 

•  Specialized  technical  workbench  areas  -  (work  areas  designed  for  tasks 
utilizing  specialized  equipment  or  technology)  e.g.  electronic  test  benches, 
simulation  and  modeling  computers,  video  analysis  equipment 

•  Non-Technical  workbench  areas  -  (variable  use  work  areas  designed  for  a 
variety  of  tasks,  not  restricted  to  specific  equipment  or  technology)  e.g.  open 
laboratory  benches  for  general  material  and  equipment  handling,  parts  and 
inventory. 

Daily  laboratory  activity  consists  of  scientists  and  technicians  utilizing  the  necessary 

equipment  and  facilities  to  perform  RDT&E  tasks.   This  connection  between  laboratory 

activity  and  equipment  utilization  could  provide  an  indication  of  capacity  utilization  for 

individual  laboratories.   However,  there  is  no  standard  format  established  to  account  for 

equipment  use  under  the  present  system. 

'    2.  NAWCAD  Laboratory  Competencies 

The   laboratories   are   administratively    grouped   into   "competencies".      Each 
competency  represents  a  group  of  laboratories  designed  to  support  similar  mission 
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requirements.  The  competency  designations  separate  the  RDT&E  laboratories  into  two 
primary  areas:  (1)  Research  and  Engineering,  and  (2)  Test  and  Evaluation.  The  facilities 
primarily  supporting  Research  and  Engineering  are  labeled  4.0  competency  laboratories, 
and  the  facilities  primarily  supporting  Test  and  Evaluation  are  labeled  5.0  competency 
laboratories.  Although  some  work  is  shared  between  competencies,  the  laboratories 
operate  independently  and  report  to  their  own  competency  managers.  Of  the  500 
RDT&E  laboratories,  350  are  identified  with  the  Research  and  Engineering  (4.0) 
competency.  The  data  for  this  thesis  were  collected  from  a  subset  of  the  Research  and 
Engineering  (4.0)  competency  laboratories.  The  operating  characteristics  of  these 
laboratories  are  described  in  the  following  section. 


B.         NAWCAD  RESEARCH  AND  ENGENEERING  (4.0)  COMPENTENCY 
LABORATORIES 

The  primary  function  of  Research  and  Engineering  (4.0)  competency  laboratories 
is  to  provide  basic  research,  applied  research,  troubleshooting,  and  engineering  support 
for  DOD  mission  requirements.2  (Collier,  1998)  Other  Research  and  Engineering  (4.0) 
laboratory  functions  include  troubleshooting,  engineering,  and  life  cycle  support  for 
existing  fleet  assets. 


2  DOD  defines  basic  research  as  efforts  typically  performed  in  laboratories  as 
experiments  to  explore  the  basic  laws  of  science  and  their  potential  application  to  DOD 
weapon  systems  or  technology  development.  Applied  research  is  research  concerned 
with  the  practical  application  of  knowledge,  material,  and/or  techniques  directed  toward  a 
solution  to  an  existent  or  anticipated  military  requirement.  (GAO,  1998,  p.  87) 
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1.  Research  and  Engineering  (4.0)  Laboratory  Activity 

The  laboratories  employ  a  variety  of  scientists  and  technicians  with  expertise  in 
specific  areas  needed  to  support  the  laboratory  functions.  Each  individual  laboratory  may 
perform  any  or  all  of  the  functions  previously  described.  The  unique  capabilities  of  each 
laboratory  are  utilized  to  meet  specific  RDT&E  project  and  mission  requirements.  The 
types  of  laboratory  activities  involved  in  meeting  these  requirements  vary  depending  on 
the  specific  project  requirements  for  a  laboratory  at  a  specific  point  in  time.  Based  on 
interviews  with  laboratory  managers  (Collier  and  Harris,  1998)  and  observation  of 
Research  and  Engineering  (4.0)  laboratory  activity,  the  following  common  types  of 
activities  were  identified: 

•  Research  and  Development  activity  -  Laboratory  conducts  basic  and  applied 
research,  tasks  are  generally  not  well  defined,  nor  repetitive. 

•  Certification  activity  -  Laboratory  is  required  to  validate  an  aircraft  system 
prior  to  flight,  flight  clearance,  or  fleet  use. 

•  In-Service  Support  activity  -  Laboratory  is  used  primarily  to  provide  direct 
fleet  support,  such  as  troubleshooting  and  correcting  existing  hardware  and 
software  problems  reported  by  the  fleet. 

•  Production  activity  -  Laboratory  activity  consists  of  repetitive,  defined  tasks. 

•  Software  Support  activity  -  Laboratory  provides  programming  and  ADP 
support. 

Categorizing  laboratories  by  the  types  of  activity  performed  would  be  useful  when 
comparing  capacity  utilization  and  financial  performance  of  individual  laboratories. 
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Presently,  these  activity  characteristics  are  not  used  to  organize  or  categorize  laboratories 
for  performance  comparison. 

2.  Research  and  Engineering  (4.0)  Laboratory  Organization 

The  Research  and  Engineering  (4.0)  competency  laboratories  are  organized  into 

three  distinct  levels,  aligning  capabilities  with  primary  RDT&E  mission  areas.   The  first 

level,  Level  I,  divides  the  350  Research  and  Engineering  (4.0)  laboratories  into  the 

following  primary  RDT&E  mission  areas: 

Air  Vehicle  RDT&E  Facilities 

Aircrew  Systems  RDT&E  Facilities 

Air  Platform  Interface  (API)  RDT&E  Facilities 

Avionics  RDT&E  Facilities 

Mission  System  RDT&E  Facilities 

Propulsion  Systems  RDT&E  Facilities 

Ship  and  Shore  Electronic  Systems  RDT&E  Facilities 

Training  Systems  RDT&E  Facilities 

Laboratories  belonging  to  the  Level  I,  Aircrew  Systems  RDT&E  Facilities  mission  area 
were  chosen  as  the  test  group  to  provide  data  for  this  thesis.  The  Aircrew  Systems 
laboratories  are  a  subset  of  the  Research  and  Engineering  (4.0)  competency  group  and  are 
identified  as  the  4.6  series  of  laboratories.  Forty-six  individual  4.6  laboratories  are 
assigned  to  the  Aircrew  Systems  RDT&E  Facilities  mission  area. 

The  second  level,  Level  II,  subdivides  each  Level  I  mission  area  into  categories  of 
functionally  similar  laboratories.  For  example,  within  the  Level  I  category,  Aircrew 
System  RDT&E,  there  are  nine  different  Level  II  functional  categories  (see  Table  2.1). 
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LEVEL 

I 

Aircrew 

Systems 

RDT&E  Facilities 

LEVEL  II 

1. 

Advanced  Crewstation  Technology  Labs 

2. 

Aircraft  Integration  /  Test  Labs 

3. 

Aircrew  Altitude  Protection  &  Breathing  System 

Facility 

4. 

Aircrew  Protection  and  Survival  Equipment  RDT&E 

5. 

Crashworthy  System  RDT&E  Facility 

6. 

Crew  Systems  Integration  Labs 

7. 

Crewstation  Transparency  and  Lighting  Labs 

8. 

Escape  System  RDT&E  Facility 

9. 

Thermophysiology  Research  Facility 

Table  2.1  4.6  Level  II  Laboratory  Categories 
Each  of  the  Level  II  laboratory  groups  listed  in  Table  2.1  provides  support  to  meet  Level 
I  Aircrew  System  mission  requirements. 

The  third  level,  Level  III,  identifies  each  individual  laboratory  by  its  specific  area 
of  expertise.  An  example  of  the  Level  II  group  is  Advanced  Crewstation  Technology 
Laboratories,  which  includes  nine  individual  (Level  III)  laboratories,  each  with  its  own 
dedicated  space  and  equipment.  Table  2.2  provides  an  example. 


Level  II 

Advanced  Crewstation  Technology  Labs 

Level  III 

1. 

Advanced  Technology  Crew  Station  -  JSF 

2. 
3. 
4. 
5. 
6. 

Cockpit  Crewstation  Integration  Facility 
CTL  Computer  Operations  Facility 
CTL  Data  Reduction  &  Task  Analysis  Lab 
CTL  Helmet  Mounted  Display/Cueing  Facility 
CTL  Mission  Control  Center 

7. 
8. 
9. 

CTL  Video  Extraction  and  Activity  Recording  Labs 
Laboratory  Instrumentation  Storage 
Man  Machine  Integration  Lab 

Table  2.2  4.6  Level  III  Laboratories 
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The  Level  III  laboratories  are  individual  entities,  coordinating  operations  with 
their  Level  II  group  of  laboratories.  Each  Level  III  laboratory  exists  as  either  a  stand- 
alone laboratory,  capable  of  full  process  completion  of  RDT&E  tasks,  or  as  a  small 
technical  support  space,  integrated  as  a  technical  component  of  its  Level  II  laboratory 
function.  This  distinction  is  not  made  clear  under  the  existing  laboratory  structure,  but 
may  be  important  when  comparing  laboratory  productivity  at  the  Level  III  organizational 
level. 

The  focus  of  this  thesis  is  centered  on  the  Aircrew  Systems  (4.6)  Level  II 
laboratories.  The  data  collected  from  these  laboratories  will  be  applied  to  the  capacity 
utilization  and  productivity  models  and  techniques  discussed  in  Chapters  V  and  VI.  The 
Aircrew  Systems  RDT&E  facilities  represent  one  segment  of  the  overall  NAWCAD 
RDT&E  laboratory  organization.  The  nine  Aircrew  Systems  (4.6)  Level  II  laboratory 
groups,  consisting  of  46  individual  Level  III  laboratories,  have  laboratory  activity 
characteristics  similar  to  the  67  Research  and  Engineering  (4.0)  competency  Level  II 
laboratories.  These  characteristics  are  also  similar  to  those  found  in  the  Test  and 
Evaluation  (5.0)  competency  laboratories,  but  due  to  the  limited  focus  of  this  research,  a 
comprehensive  description  of  (5.0)  laboratory  activity  will  not  be  presented. 

3.  Research  and  Engineering  (4.0)  Laboratory  Financial  Management 

The  accounting  and  financial  management  of  Research  and  Engineering  (4.0) 
laboratory  activity  is  another  element  of  the  organization  that  must  be  defined  to  ensure 
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proper  measurement  of  performance.  The  Aircrew  Systems  (4.6)  laboratories  are  funded 
through  the  Navy  Working  Capital  Fund  (NWCF).  Each  of  the  Aircrew  Systems  (4.6) 
laboratories  is  set  up  as  a  NWCF  account  and  must  plan  and  execute  its  yearly  budget 
based  on  expected  levels  of  customer/program  requirements  and  operating  costs.  The 
goal  is  to  achieve  zero  net  gain  or  loss  at  the  end  of  the  fiscal  year  when  comparing  actual 
laboratory  revenue  with  all  costs  allocated  to  the  NWCF  laboratory  account.  Revenue  is 
generated  from  the  fees  charged  to  paying  customers  of  laboratory  services.  Laboratory 
services  are  primarily  used  by  DOD  program  sponsors,  but  services  are  also  available 
through  contract  agreements  for  non-DOD  customers  requiring  the  unique  RDT&E 
capabilities  that  the  NAWCAD  facilities  can  provide.  (Collier,  1998) 

Presently,  the  Research  and  Engineering  (4.0)  competency  facilities  do  not  receive 
congressionally  appropriated  funds  other  than  NWCF  account  dollars.  The  Test  and 
Evaluation  (5.0)  competency  facilities  do  receive  appropriated  Major  Range  and  Test 
Facility  Base  (MRTFB)  funds,  which  provide  supplemental  funding  for  federal  RDT&E 
activities  determined  to  be  critical  DOD  or  national  assets.  The  amount  of  MRTFB  funds 
distributed  to  the  Test  and  Evaluation  (5.0)  competency  laboratories  in  FY1997  was  $80 
million.  MRTFB  funds  are  intended  for  any  DOD  asset  meeting  congressional 
specifications  and  are  not  specifically  restricted  to  Test  and  Evaluation  (5.0)  laboratories. 
It  has  been  policy,  set  at  the  local  (NAWCAD)  level,  not  to  fund  Research  and 
Engineering  (4.0)  competency  laboratories  with  MRTFB  funds.  (Runion,  1998) 
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Although  not  all  of  the  NAWCAD  laboratories  are  presently  funded  as  NWCF 
accounts,  there  are  indications  in  the  Defense  Reform  Initiative  that  accounting  and 
financial  management  of  all  RDT&E  facilities  will  be  standardized  to  conform  to  best 
business  practices  of  full-cost  accounting.  The  NWCF  is  identified  as  the  DOD  financial 
management  system  most  capable  of  achieving  these  accounting  goals.  (Defense  Reform 
Initiative,  1997)  The  Aircrew  Systems  (4.6)  laboratories  NWCF  account  structure  is 
being  reviewed  by  management  as  the  potential  standard  for  all  RDT&E  laboratories,  and 
initiatives  have  recently  been  put  in  place  to  convert  other  NAWCAD  laboratories  to  the 
NWCF  budget  accounting  system  (Collier,  1998). 

The  Aircrew  Systems  (4.6)  laboratories  operating  as  NWCF  accounts  charge 
customers  a  fee  for  laboratory  services  and  are  classified  as  Rated  Service  Accounts 
(RSA).  RSA  laboratories  charge  customers  for  laboratory  services  based  on  a  pre-set 
hourly  laboratory  rate.  Rates  are  determined  on  a  yearly  basis,  calculated  by  using 
projected  levels  of  demand  for  laboratory  activities  and  assigning  an  hourly  rate  that  will 
produce  a  total  revenue  amount  equal  to  the  expected  laboratory  costs  assigned  for  the 
year.  The  rates  are  set  and  reviewed  during  the  year  prior  to  execution  through  an 
internal  budget  review  process,  and  are  published  as  a  constant  -rate  for  the  entire 
execution  year.  Under  the  existing  accounting  system,  the  costs  assigned  to  laboratory 
activities  consist  of  direct  operating  costs  and  do  not  include  indirect  cost  items  such  as 
production  overhead  and  general  and  administrative  costs.  Table  2.3  identifies  the  types 
of  costs  used  to  determine  FY1997  RSA  laboratory  hourly  rates. 
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LAB    4.X    1997    OPERATING    COSTS 

Maintenance 

Operations 

Contracts                          $113,000 

Contracts                      $116,000 

Maintenance                     $    40,600 

Consumables               $    15,000 

Travel                               $      1,600 

Travel                              $    14,700 

Labor                                    $183,000 

Labor                                $206,000 

Training                       $      9,000 

Utilities                     $      1,600 

Total   Maint                     $338,200 

Total    Ops                      $362,300 

Table  2.3  Example  of  Costs  assigned  for  laboratory  rate  determination 


The  calculated  rates,  therefore,  reflect  the  projected  amount  needed  to  recover  all 
direct  costs  associated  with  the  laboratory,  but  do  not  account  for  indirect  costs  charged 
to  overall  NAWCAD  facilities.  In  FY1997,  NAWCAD  RDT&E  indirect  overhead  and 
general  and  administrative  costs  totaled  $140  million  and  accounted  for  one  quarter  of  all 
costs  attributed  to  NAWCAD  RDT&E  facilities.  (Runion,  1997)  In  recognition  of  the 
DOD  directive  toward  full-cost  accounting  of  activities,  these  indirect  costs  will  be 
incrementally  added  to  the  assigned  laboratory  costs  over  the  next  three  years.  The 
increase  in  the  cost  base  for  RSA  laboratories  will  drive  laboratory  rates  up,  assuming  all 
other  variables  are  held  constant.  Increased  rates  may  drive  demand  for  laboratory 
services  down,  which  would  subsequently  provide  pressure  for  additional  rate  increases 
to  cover  the  loss  in  volume  of  activity.  This  self-perpetuation  of  rate  increases  is  a 
potential  problem  facing  NAWCAD  RDT&E  laboratories  as  they  shift  to  a  full-cost 
system.  (Runion,  1998) 

Some  if  not  all  of  the  additional  costs  allocated  to  laboratories  can  be  countered 
with  improvements  in  laboratory  efficiency  and  productivity.    Reducing  infrastructure 
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and  improving  laboratory  productivity  are  two  methods  identified  by  facilities  managers 
to  better  manage  available  resources  and  reduce  costs.  (Collier,  1998)  Accurate  measures 
of  capacity  utilization  and  productivity  are  necessary  for  management  to  focus  on 
appropriate  business  decisions  influencing  the  efficient  use  of  resources.  The  next 
chapter  reviews  potential  models  and  measures  of  capacity  utilization  and  productivity 
followed  by  identification  of  models  and  measures  suitable  for  the  NAWCAD  RDT&E 
laboratory  organization. 
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III.     CAPACITY  UTILIZATION  AND  PRODUCTIVITY 


A.         LITERATURE  REVIEW 

Business  scholars  and  industry  professionals  have  generated  extensive  research  in 
efforts  to  determine  levels  of  capacity  and  proper  methods  of  measuring  and  accounting 
for  capacity  utilization.  Effective  management  and  efficient  use  of  capacity  are  integral 
components  of  production  oriented  industry;  there  is  abundant  literature  published  in  the 
area  of  capacity  utilization  in  production  manufacturing.  A  search  of  industry  capacity 
related  literature  published  since  1980  found  over  300  articles  focused  on  capacity 
utilization  in  production  and  manufacturing  environments.  The  methods  of  measuring 
output  as  an  indicator  of  capacity  utilization  in  manufacturing  industry  are  relatively  well 
defined  and  have  become  accepted  as  business  tools  necessary  to  compete  in  today's 
manufacturing  marketplace.  Nonetheless,  capacity  and  capacity  management  remain 
essentially  elusive  concepts.  No  single  tool  or  single  view  of  capacity  management  is 
best.  (McNair  and  Vangermeersch,  1 996) 

1.  Research  and  Development  (R&D)  Industry  Review 

No  standards  could  be  found  to  measure  capacity  utilization  and  performance  of 
research  and  development  (R&D)  activities.  A  variety  of  different  approaches  to 
measuring  capacity  utilization  were  found,  but  few  have  been  universally  accepted,  and 
none  have  been  established  as  a  standard  for  R&D  activity.    A  1996  Conference  on 
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Performance  Measurements  for  R&D,  organized  by  the  International  Quality  and 
Productivity  Centre,  Central  Research  Laboratories,  Middlesex,  UK,  concluded  that  there 
are  few  universal  measures  and  that  each  company  must  select,  from  a  wide  array  of 
measures  and  approaches,  a  configuration  that  matches  the  requirements  of  each  situation 
(Nixon,  1997).  The  conference  report  reviewed  presentations  given  by  three  R&D 
industry  management  consultants,  and  summarized  the  content  of  nine  papers  written  by 
a  variety  of  companies  focused  on  the  implementation  and  operation  of  R&D 
performance  measurement  systems.  (Nixon,  1 997) 

The  conference  report  confirms  that  measurement  of  R&D  production  and 
capacity  utilization  is  complex  and  difficult  to  define  and  quantify  (Nixon,  1997).  The 
nine  papers  presented  at  the  conference,  based  on  practices  in  Corporate  Research 
Laboratories,  emphasize  that  there  is  increased  interest  in  performance  measurements  for 
R&D.  Yet,  no  one  universal  approach  exists.  Each  organization  has  developed  a  unique 
method  achieving  varying  degrees  of  success.  One  consistent  problem  is  the  difficulty  in 
defining  output,  exemplified  in  the  following  excerpt  from  the  conference  report: 
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Richard  Duggan,  Senior  Advisor  on  Innovation,  DTI,  related  his 
experience  as  head  of  the  Unilever  Research  Laboratory  when  he  was 
challenged  to  improve  output  by  15%.  The  first  problem  was  to  define 
output;  MIT's  advice  to  him  was  that  it  was  impossible  to  measure  the 
output  of  R&D  but  that  it  was  possible  to  monitor  whether  the  most 
important  parameters  influencing  output  were  improving  or  not.  Changes 
could  be  measured  and  it  was  decided  to  measure  and  manage  the  use  of 
scientists'  time,  working  space  and  project  completion  time  in  order  to 
bring  about  the  required  R&D  output  improvement.  The  programme 
achieved  its  goals,  and  the  value  of  the  space  saved  (with  the  help  of 
interior  space  architect,  David  Leon)  funded  the  implementation.  (Nixon, 
1997) 


2.  General  Accounting  Office  Report  on  RDT&E  Infrastructure 

Other  examples  of  industry  practices  are  detailed  in  a  General  Accounting  Office 
(GAO)  report  published  in  January  1998  titled  Best  Practices,  Elements  Critical  to 
Reducing  Successfully  Unneeded  RDT&E  Infrastructure,  which  analyzed  approaches 
used  by  organizations  outside  of  the  federal  government  to  realign  RDT&E  infrastructure. 
The  GAO  report  states  that  a  clear  relationship  exists  between  the  recent  trend  in  industry 
of  restructuring  and  reengineering  and  the  need  for  accurate  capacity  utilization  and 
productivity  measures.  GAO  examined  restructuring  efforts  by  two  organizations-the 
Boeing  Company  Defense  &  Space  &  Defense  Systems  Group  and  the  Defence  Research 
Agency  within  the  British  Ministry  of  Defence-both  of  which  reduced  substantially  their 
laboratories'  infrastructure  and  costs.  (GAO,  1998) 

The  approach  taken  by  each  organization  included  several  common  elements. 
Both  organizations  (1)  developed  core  missions  and  aligned  them  with  their  customers' 
needs,  (2)  determined  what  infrastructure  they  had  and  how  it  supported  their  missions, 
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and  (3)  collected  accurate,  reliable,  and  comparable  data  about  their  facilities  across-the- 
board  to  reduce  confusion,  prevent  facility  officials  from  claiming  they  should  be 
exempted  from  restructuring,  and  reduce  their  assertions  that  the  facilities  were  unique  or 
incomparable.  One  of  the  critical  elements  identified  as  a  key  to  their  success  was 
accurate,  reliable,  and  comparable  data  that  captured  total  infrastructure  costs  and 
utilization  rates  for  each  affected  activity.  (GAO,  1998) 

Both  Boeing  and  the  British  Defence  Research  Agency  discovered  that  their 
financial  management  systems  could  not  capture  or  evaluate  either  the  total  costs  of 
operating  their  labs  or  the  facility  utilization  rates.  Because  accurate,  reliable,  and 
comparable  data  on  infrastructure  costs  and  utilization  rates  were  critical,  both 
organizations  developed  standardized  data  collection  instruments  to  capture  necessary 
details  about  their  infrastructure.  The  Boeing  Company  Defense  &  Space  Group 
included  details  about  laboratory  product  areas,  unique  capabilities,  equipment  values, 
utilization  rates,  maintenance  costs,  personnel  costs  and  capabilities,  anticipated 
capability  requirements,  and  potential  consolidation/closing  requirements.  (GAO,  1998) 

The  data  collected  were  analyzed  by  laboratory  function.  Functional  categories 
tied  each  laboratory's  activities  to  its  primary  mission.  Brainstorming  sessions  listed  45 
to  50  functions,  which  were  winnowed  to  15  prime  functions.  Categorization  of 
laboratories  into  functional  groups  enabled  the  data  to  be  compared  among  laboratories  of 
similar  function. 
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The  question  of  how  to  capture  accurate  utilization  rate  data  is  not  addressed  in 
the  report.  Unfortunately,  the  specific  details  of  Boeing's  effort  in  developing  utilization 
measurement  devices  have  not  been  made  available  to  sources  outside  of  the 
organization.  Although  the  report  does  not  specify  the  metrics  used  to  measure 
utilization  rates,  it  does  emphasize  that  multidisciplinary  review  teams  were  used  to 
validate  and  analyze  the  data.  The  review  teams  included  scientists,  strategic  planners, 
financial  experts,  accountants,  engineers,  and  laboratory  operations  specialists.  This 
cross  section  of  organizational  expertise  ensured  that  accurate  and  comparable  data  were 
being  obtained  and  allowed  laboratory  personnel  to  participate  in  the  data  collection 
process.  (GAO  1998) 

3.  National  Aeronautical  and  Space  Administration  (NASA) 

Specific  examples  of  capacity  utilization  metrics  designed  for  RDT&E  activities 
are  found  in  methods  used  by  the  National  Aeronautical  and  Space  Administration 
(NASA)  laboratories.  (EMA,  1998)  Table  3.1  identifies  utilization  criteria  for  their 
RDT&E  facilities,  including  Facility  Type,  Unit  of  Utilization  Measure,  and  Baseline  Use 
Measure.  NASA  has  incorporated  the  concept  of  Equivalent  Utilization  Days  (EUD)  for 
certain  types  of  facilities  as  a  standard  against  which  to  measure  actual  utilization.  For 
example,  a  baseline  of  220  EUD  days  per  year  represents  the  number  of  EUD  days  that 
the  facility  is  available  for  testing,  including  time  for  test  article  prep,  test  ops,  and  tear 
down,  but  excluding  adverse  weather  impacts,  normal  maintenance  and  other  down  time. 
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The  actual  usage  is  compared  to  the  baseline  of  220  EUD  days  and  presented  as  a  percent 
of  baseline.  (Smith,  1998) 


NASA  FACILITIES  UTILIZATION  CRITERIA 

FACILITY  TYPE 

UNIT  OF  UTILIZATION  MEASURE 

BASELINE  USE  MEASURE 

Wind        Tunnels,        large 
vacuum    chambers,    flight 
simulators,      engine      test 
facilities  and  other  research 
and  development  and  test 
facilities 

Equivalent  Utilization  Days  (EUD)  facility 
was  occupied  for  testing.   An  EUD  =  one 
8  hr  shift;  maximum  EUD/Day  =  3.    This 
unit  of  measure  is  devised  to  show  use  in 
flexible    but    uniform    terms   -    regular, 
periodic,  or  varying  shift  operations  versus 
the  baseline. 

220  EUD  days  per  year  facility  is 
normally  available  for  testing 
including  time  for  test  article  prep,  test 
ops,  and  tear  down,  but  EXCLUDING 
adverse  weather  impacts,  normal 
maintenance  and  other  down  time. 

Laboratories 

An  assessment  of  the  level-of-use  or  need 
for     the     housed     lab     equipment,     or 
population  housed  in  lab  area. 

100%  if  all  equipment  is  used  at  least 
seasonally  or  is  needed  for  future 
activities,  or  if  more  appropriate,  rated 
population  @  300  Net  Square  Feet  per 
Person 

Computer  Facilities 

EUD  days  the  facility  housed  an  active 
ADP  operation. 

260  EUD/year  facility  is  normally 
available  to  support  ADP  operations 

Table  3.1  NASA  Facilities  Utilization  Criteria  (From  EMA,  1998) 

In  other  laboratory  settings,  NASA  uses  a  separate  metric.  An  assessment  of  the 
level-of-use  or  need  for  the  housed  lab  equipment  is  used  as  the  Unit  of  Utilization 
Measure.  The  baseline  for  laboratory  equipment  utilization  is  set  at  100  percent  as  long 
as  all  equipment  in  the  lab  is  used  at  least  seasonally  or  is  needed  for  future  activities. 
The  lab  assessment  of  actual  use  of  equipment  is  reported  as  a  percent  of  the  baseline. 
Details  are  not  provided  about  NASA's  assessment  techniques  for  determining  level-of- 
use  of  laboratory  equipment.  (EMA,  1 998) 
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4.  Office  of  the  Inspector  General  Audit  Report  on  DOD  Resource 

Utilization  (RUMS) 

Another  application  of  RDT&E  associated  utilization  metrics  is  found  in  a  1 995 

Office  of  the  Inspector  General  Audit  Report  on  DOD  Resource  Utilization.  (DOD,  1995) 

The  Resource  Utilization  Measurement  System     (RUMS)  was  designed  to  capture 

utilization  rates  of  major  DOD  Test  ranges  in  response  to  1995  Base  Realignment  and 

Closure  (BRAC)  requirements.   The  equations  used  to  measure  utilization  and  efficiency 

are  based  on  ratios  of  resource  availability  and  use  as.  described  below: 

UTILIZATION  =  (C  +  D)/(A  +  D) 

EFFICIENCY  =  (A  -  B)/A 

A  =  Time  the  resource  was  made  available/staffed  to  support  paying  customers 
under  actual  staffing  conditions.  It  does  not  include  excess  use  above 
budgeted  capacity.  But  it  does  include  unplanned  lost  time  due  to  weather 
and  periods  of  non-use  if  people  were  available  in  a  pay  status  to  operate  the 
facility. 

B  =  Unscheduled  Non- Availability  due  to  external  constraints  (i.e.  weather, 
unscheduled  maintenance,  etc.). 

C  =  Use  of  a  resource  paid  for  by  a  customer  (includes  set-up  and  teardown,  if 
they  preclude  use  of  the  resource  by  another  customer). 

D  =  Use  of  a  resource  in  excess  of  the  normal  resource  budgeted  capacity. 

Example: 


Hours 


A  = 

1500 

B  = 

30 

C  = 

1400 

D  = 

500 

UTILIZATION  =  (1400+500)/(1500+500)  =  95% 
EFFICIENCY   =      (1500-30)71500  =  98% 
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The  methods  of  measuring  utilization  rates  used  in  the  audit  report  were 
developed  specifically  for  Test  Ranges  and  did  not  address  other  RDT&E  facilities.  No 
published  references  were  found  noting  application  of  the  RUMS  equations  for  RDT&E 
laboratory  settings.  It  is  also  important  to  note  that  the  audit  team  experienced  difficulty 
in  obtaining  accurate,  reliable  data  from  the  test  sources  for  their  study  (DOD,  1995). 
Similar  problems  were  cited  in  the  cases  of  Boeing  and  the  British  Defence  Research 
Agency  (GAO,  1998).  The  financial  management  system  could  not  accurately  capture 
the  total  cost  of  operating  or  the  facility  utilization  rate  data  required.  (DOD,  1995) 


B.         BENCHMARK  APPLICATION 

The  lack  of  research  and  development  industry  standards  of  performance  and 
measurement  reduces  the  potential  for  benchmark  applications  for  NAWCAD  RDT&E 
laboratories.  The  inconsistency  of  methodology  and  lack  of  available  industry  specific 
information  about  capacity  utilization  metrics  limits  detailed  review.  The  Boeing 
RDT&E  laboratory  organization  displays  the  highest  degree  of  similarity  with  the 
NAWCAD  RDT&E  labs.  A  more  detailed  review  of  the  capacity  utilization  metric 
developed  by  Boeing  may  provide  potential  benchmark  measures  for  NAWCAD  and 
other  DOD  laboratories.  In  the  absence  of  compatible  RDT&E  benchmark  measures, 
NAWCAD  should  establish  internal  standards  of  performance  for  laboratory  capacity 
utilization  and  productivity.     The  following  chapters  present  potential  models  and 
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techniques  for  measuring  capacity  utilization  and  productivity  of  NAWCAD  RDT&E 
facilities. 
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IV.      REVIEW  OF  CAPACITY  UTILIZATION  MODELS 


A.         INDUSTRY  CAPACITY  UTILIZATION  MODELS 

No  single,  magical  capacity  number  will  work  in  all  companies,  all  settings  or  all 

decision  contexts.  Rather,  an  overall  philosophy  or  approach  to  capacity  supports  a 
company's  efforts  to  improve  performance  through  better  management  and  utilization  of 
its  resources  (McNair  and  Vangermeersch,  1996).  The  financial  and  accounting 
professions  have  produced  a  variety  of  models  addressing  capacity  utilization  with  a 
focus  on  effectively  managing  the  cost  of  capacity.  The  capacity  utilization  and  cost 
models  are  designed  to  provide  management  with  the  necessary  tools  to  achieve 
maximum  utilization  of  company  resources.  Consisting  of  a  set  of  action-based  tools  for 
making  products  and  providing  better,  faster  and  cheaper  services  to  customers,  the 
development  of  capacity  management  systems  is  synonymous  with  best  management 
practice  in  management  accounting  (McNair  and  Vangermeersch,  1996). 

Applications  of  concepts  utilized  by  the  existing  capacity  utilization  and  cost 
models  appear  to  have  the  potential  to  enhance  the  development  of  a  capacity 
management  system  for  NAWCAD  RDT&E  laboratories.  Table  4.1  provides  a  list  of 
existing  models  identified  in  the  literature  and  selected  attributes  of  the  models.  A  review 
of  the  capacity  management  models  and  their  respective  capacity  cost  measurement  tools 
and  techniques  is  presented  in  this  chapter. 
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----^Features 
Model    "--«^. 

Capacity  Baseline 
Emphasized 

Primary  Time  Frame 
of  Analysis 

Organizational 
Focus 

Resource 
Effectiveness  Model 

Theoretical  Capacity 

Short-to  long-term 

Process/Plant/ 
Company  Levels 

Capacity  Utilization 
Model 

Theoretical  Capacity 

Short-  to 
Intermediate-Term 

Process/Plant/ 
Company  Levels 

Capacity  variance 
Model 

Theoretical  Capacity 

Short-  to 
Intermediate-Term 

Process/Plant  Levels 

CAM- 1  Capacity 
Model 

Theoretical  Capacity 

Short-  to  Long-Term 

All  Levels 
(Potential) 

CUBES  Model 

Theoretical  Capacity 

Short-  to  Intermediate-Term 

Process/Plant/  Company 
Levels 

Cost  Containment 
Model 

Implicit  Theoretical 
Capacity 

Intermediate-Term 

All  Levels 
(Potential) 

Gantt  Idleness 
Charts 

Practical  Capacity 

Short-term 

Process  Level 

Supplemental  Rate 
Method 

Practical  Capacity 

Short-term 

Process/Plant  Levels 

Theory  of 

Constraints 

Capacity  Model 

Practical  Capacity 
(Marketable) 

Short-  to 
Intermediate-Term 

Process/Plant/ 
Company  Levels 

Normalized  Costing 
Approach 

Normal  Capacity 

Intermediate-Term 

Process/Plant  Levels 

ABC  and  Capacity 
Cost  Measurement 

Normal  Capacity 

Short-  to 
Intermediate-Term 

Process/Plant/ 
Company  Levels 

Integrated  TOC- 
ABC  Model 

Various 

Short-  to 
Intermediate-Term 

Process/Plan  t/Value 
Chain  Levels 

Table  4.1  Tools  and  Techniques  for  Measuring  the  Cost  of  Capacity 
(From  McNair  and  Vangermeersch,  1996) 


Each  capacity  utilization  model  and  cost  measurement  tool  listed  takes  a  different 

approach  to  measuring  the  utilization  and  cost  of  capacity.    These  models  are  grouped 

according  to  the  capacity  baseline  measure  they  emphasize:  theoretical,  practical,  or 

normal  capacity.  A  definition  of  each  of  these  baseline  measures  is  listed  below: 

•     Theoretical  capacity  -  the  optimal  amount  of  work  that  a  process  or  plant  can 
complete  using  a  24-hour,  seven-day  operation  with  zero  waste,  i.e.,  the 
maximum  output  capability,  allowing  no  adjustment  for  preventive 
maintenance,  unplanned  downtime,  and  shutdown. 
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Practical  capacity  -  the  level  of  output  generally  attainable  by  a  process,  i.e., 
theoretical  capacity  adjusted  downward  for  unavoidable  nonproductive  time: 
such  as  set-ups,  maintenance  or  breakdowns. 

Normal  capacity  -  the  average,  expected,  utilized  capacity  of  a  machine, 
process  or  plant/unit  over  a  defined  period  of  time  (day,  week,  month,  year). 


Another  important  dimension  of  capacity  cost  management  is  organizational 
focus.  The  issues  impacting  capacity  cost  management  at  the  process  level  often  differ 
from  those  faced  at  the  plant  or  company  level.  Each  model  focuses  on  one  or  more  of 
the  following  organizational  levels:  (1)  process  level,  (2)  plant  or  sub-unit  level,  (3) 
company  level,  and  (4)  value  chain  level.  (McNair  and  Vangermeersch,  1996)  An 
organization  and  its  existing  structure  and  capabilities  define  the  first  three  levels.  The 
process  level,  which  can  range  from  one  task  to  an  assembly  line,  focuses  on  individual 
units  of  output.  The  plant  or  sub-unit  level  suggests  several  processes  and  several  unique 
types  of  outputs.  At  the  company  level  or  strategic  unit,  many  different  plants  or  sub- 
units  combine  to  create  a  complex  organization  that  serves  many  markets  with  many 
different  types  of  products  and  services.  Finally,  the  value  chain  level  shifts  its  attention 
to  all  of  the  activities  and  resources  of  all  organizations  used  to  bring  a  good  to  the 
consumer. 

The  time  frame  of  decision  analysis  is  another  factor  to  consider  when  reviewing 
different  models.  Model  assumptions  about  the  length  of  time  involved  in  affecting 
capacity  utilization  issues  have  a  direct  impact  on  an  organization's  ability  to  change  the 
cost  and  management  of  its  capacity.    In  the  short  run,  theoretical  capacity  is  constant; 
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very  little  can  be  done  to  change  the  theoretical  capacity  of  a  process.    The  focus  in  the 
short-run  model  is  on  improving  the  utilization  of  existing  resources  and  processes. 

As  the  time  frame  extends  to  the  intermediate  term,  an  organization  can  act  to 
change  how  the  process  operates,  impacting  the  theoretical  capacity  of  the  process, 
without  changing  the  physical  structure  of  the  process.  The  focus  in  the  intermediate-run 
model  shifts  to  maximizing  the  flexibility  of  existing  processes  in  order  to  decrease  future 
investment  requirements.  Finally,  in  the  long-run,  a  wide  range  of  techniques  and 
measures  can  be  used  to  adjust  capacity  and  its  utilization.  (McNair  and  Vangermeersch, 
1996) 

The  issues  of  time  frame  of  analysis,  organizational  focus,  and  baseline  capacity 
measures  are  dimensions  that  can  be  used  to  select  a  model  from  among  the  various 
capacity  cost  measurement  models.  Matching  these  model  characteristics  to 
organizational  objectives  will  help  guide  the  selection  process.  Other  characteristics 
common  to  capacity  management  models  are  classifications  of  use  or  deployment  of 
capacity. 

Understanding  the  terms  used  to  describe  capacity  deployment  is  essential  to 

building  successful  utilization  measures  and  communicating  their  results  to  management. 

The  following  categories  are  commonly  used  in  the  models  to  break  down  overall 

capacity  into  specific  types  of  deployment. 

•     Productive  capacity  -  capacity  that  provides  value  to  the  customer.  Productive 
capacity  is  used  to  produce  a  product  or  provide  a  service.  It  is  based  on  the 
theoretical,  or  maximum,  value-creating  ability  of  the  company's  resources 
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Nonproductive  capacity  -  capacity  neither  in  a  productive  state  nor  in  one  of 
the  defined  idle  states.  Nonproductive  capacity  includes  setups,  maintenance 
and  scrap 

Planned  nonproductive  capacity  -  capacity  planned  for  use  that  is  temporarily 
out  of  use  due  to  process  variability,  such  as  the  lack  of  materials,  machine  or 
process  breakdown,  or  delays 

Planned  idle  capacity  -  capacity  not  currently  scheduled  for  use;  planned  idle 
capacity  might  be  planning  for  preventive  maintenance 

Excess  capacity  -  permanently  idle  capacity  that  is  not  marketable  or  usable 
under  existing  operating  or  market  or  policy  conditions 

(McNair  and  Vangermeersch,  1 996) 


B.         MODEL  FEATURES  AND  COMPARISONS 

The  different  features  of  each  model  are  described  in  the  Management  Accounting 

Practices  Handbook,  Measuring  the  Cost  of  Capacity,  1 996,  and  are  summarized  in  the 
following  section  to  provide  a  general  understanding  of  model  characteristics  and  to 
facilitate  comparison  between  models: 


• 


Resource  Effectiveness  Model: 

>  Analyzes  economic  impact  of  capacity  management  decisions 

>  Assumes  that  "zero  waste"  is  the  goal 

>  Supports  decisions  across  all  time  frames 

>  Provides  an  integrated  financial  and  operational  analysis  of  resource 
decisions 

>  Recommended  for  firms  that  use  process,  cellular,  or  assembly  line 
manufacturing  methods. 

Capacity  Utilization  Model: 

>  Focuses  on  waste  as  key  capacity  measure 

>  Separates  causes  of  capacity  waste  by  time  frames  and  actionability 

>  Supports  decisions  for  short-to  intermediate-time  frames  of  analysis 

>  Consists  of  systemic  capacity  measures 
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>  Recommended  for  use  in  conjunction  with  other  continuous  improvement- 
supporting  capacity  models,  such  as  Theory  of  Constraints 

•  Capacity  Variance  Model: 

>  Details  actual  performance  against  theoretical  capacity 

>  Identifies  causes  of  capacity  losses 

>  Supports  decisions  for  short-  to  intermediate-time  frames  of  analysis 

>  Can  be  tracked  against  improvement  goals 

>  Recommended  for  companies  that  wish  to  add  some  level  of  capacity  cost 
management  reporting  to  existing  management  report  packages. 

•  CAM-I  Capacity  Model: 

>  Integrates  capacity  data  across  many  dimensions 

>  Ties  to  the  financial  reporting  system 

>  Supports  decisions  for  short-  to  long-time  frames  of  analysis 

>  Supports  and  integrates  activity-based  costing 

>  Uses  time  as  a  unifying  measure 

>  Recommended  for  companies  to  obtain  the  maximum  benefit  from  data 
warehouse/database  capabilities  to  provide  an  integrated,  flexible  reporting 
package  to  be  used  across  an  organization. 

•  CUBES  Model: 

>  Integrates  financial  and  nonfinancial  data 

>  Builds  from  activity-based  costs 

>  Supports  decisions  for  short-  to  intermediate-time  frames  of  analysis 

>  Provides  a  dynamic  analysis  and  least-cost  solution 

>  Recommended  for  companies  facing  high  capital  investment  with  short 
product  life  cycles 

•  Cost  Containment  Model: 

>  Focuses  on  support/service  costs 

>  Supports/integrates  with  activity-based  costing 

>  Supports  decisions  for  intermediate-time  frame  of  analysis 

>  Builds  on  value-added,  market-based  models 

>  Recommended  for  companies  that  are  conducting  competitive  bidding  for 
internal  services  and  benchmarking  studies  that  focus  on  costs  per 
process/activity 


38 


•  Gantt  Idleness  Charts: 

>  Supports  decisions  for  short  term  time  frame  of  analysis 

>  Focuses  on  the  process  level  of  the  organization 

>  Summarize  performance  in  operational  and  financial  terms 

>  Detail  costs  and  causes  of  idleness 

>  Recommended  for  factory  environments  using  departmental  or  cellular 
manufacturing  approaches 

•  Supplemental  Rate  Method: 

>  Supports  decisions  for  short-term  time  frame  of  analysis 

>  Focuses  on  the  process  and  plant  levels  of  the  organization 

>  Focuses  on  profit  impact  of  idleness 

>  Supports  internal  and  external  reporting 

>  Recommended  for  small  companies  with  easily  defined  capacity  costs  and 
issues 

•  Theory  of  Constraints  Capacity  (TOC)  Model: 

>  Supports  decisions  for  short-to  intermediate-time  frame  of  analysis 

>  Highlights  key  constraints  inhibiting  process  performance 

>  Useful  in  plants  or  processes  using  TOC  in  their  management  processes 

>  Recommended  for  companies  using  TOC  approaches  elsewhere  in  the 
organization  such  as  external  and  TOC-based  management  reporting 

•  Normalized  Costing  Approach: 

>  Supports  decisions  for  intermediate  time  frame  of  analysis 

>  Focuses  on  the  process  and  plant  levels  of  the  organization 

>  The  capacity  of  a  process  is  determined  using  practical  capacity  baselines 
set  over  a  three  to  five  year  period 

>  Normalized  cost  is  determined  by  combining  cost  and  capacity 
information  to  create  a  cost  estimate  under  a  given  set  of  operating 
conditions 

>  Recommended  for  complex  manufacturing  companies 

•  Activity  Based  Costing  (ABC)  and  Capacity  Cost  Measurement: 

>  Supports  short-to  intermediate-time  frame  of  analysis 

>  Fits  into  activity-based  cost  model 

>  Reports  both  the  quantity  and  cost  of  idle  capacity 

>  Strong  emphasis  on  resources 

>  Serves  as  a  bridge  between  more  conventional  views  of  capacity  cost 
management  and  ABC 
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•     Integrated  TOC-ABC  Model: 

>  Uses  mathematical  modeling  to  solve  for  optimal  capacity  utilization 

>  Focuses  on  product  mix  and  marginal  revenue 

>  Provides  a  superior  solution  to  a  pure  TOC  or  pure  ABC  methodology 
when  at  least  one  bottleneck  operation  exists 

(McNair  and  Vangermeersch,  1 996) 

An  expanded  comparison  chart  listing  features  of  the  different  capacity  management 
models  is  presented  in  Appendix  A. 

Aligning  model  attributes  with  NAWCAD  objectives  for  capacity  utilization 
management  is  an  important  step  in  the  model  selection  process.  Interviews  with 
NAWCAD  facility  and  laboratory  managers  identified  management  criteria  for 
development  of  capacity  utilization  measures.  Models  and  measures  selected  must 
provide  management  with  information  relevant  to  decision-making  in  these  areas.  The 
following  list  is  a  summary  of  issues  considered  by  management  to  be  important  in  the 
development  of  NAWCAD  RDT&E  laboratory  capacity  utilization  measures: 

Capital  investment  decision  analysis 

Business  strategy  development 

Outsourcing/Consolidation  decision  analysis 

Laboratory  facilities  space  limitations/allocation  decision  analysis 

Identify  NAWCAD  business  strengths  and  weaknesses 

Identify  full-cost  of  capacity  and  laboratory  utilization 

Integrate  with  activity-based  cost  system 

Laboratory  rate  structure  management  (rate  reduction) 

Provide  accurate,  relevant  financial  information  about  laboratory  activity 


40 


•  Establish  system  to  anticipate  and  accommodate  DOD  RDT&E  infrastructure 
reorganization  requirements  (BRAC,  Defense  Reform  Initiative) 

•  Establish  comparison  baseline  for  competing  laboratories 

•  Improve  laboratory  efficiency 

•  Determine  appropriate  laboratory  productivity  measures 

The  issues  pertinent  to  NAWCAD  RDT&E  laboratory  capacity  utilization 
highlight  a  need  for  an  organizational  focus  at  all  levels.  Information  provided  by 
measures  of  capacity  utilization  is  required  for  top-level  business  strategy  development  as 
well  as  the  operational  level  management  of  laboratory  activity.  Additionally,  the  need 
for  short-,  medium-,  and  long-term  time  frame  analysis  is  implied  by  the  need  for  short- 
term  yearly  rate  structure  management,  medium-term  requirements  associated  with  DOD 
RDT&E  infrastructure  reorganization,  and  long-term  capital  investment  decision  analysis. 
Implementation  of  full-cost  and  activity-based  cost  (ABC)  accounting  systems 
throughout  the  NAWCAD  organization  mandates  model  compatibility  with  ABC 
systems.  Changes  in  the  financial  management  system  for  NAWCAD  such  as  the 
proposed  Cost  Based  Management  Tool  (CBMT)  and  ABC  will  require  a  strong  tie 
between  the  operational  and  financial  processes  within  the  RDT&E  organization. 
Finally,  NAWCAD  facilities  managers  desire  decision-making  information  for  both  the 
planning  and  control  of  RDT&E  capacity  utilization  through  better  management 
reporting  and  analysis  of  laboratory  activity.  (Collier,  1998) 

The  characteristics  for  each  of  the  twelve  models  are  referenced  against  the 
specific  NAWCAD  RDT&E  capacity  utilization  management  objectives  in  Table  4.2.  Of 
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the  twelve  models  listed,  the  one  that  best  meets  these  criteria  and  can  be  applied  to  the 
unique  research  and  development  environment  is  the  CAM-I  Capacity  Model  developed 
by  the  Cost  Management  Systems  Program  of  the  Consortium  for  Advanced 
Manufacturing  -  International  (CAM-I)  Capacity  Group.  (Klammer,  1996) 
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Resource  Effectiveness 
Model 

X 

X 

X 

Capacity  Utilization 
Model 

X 

X 

X 

Capacity  Variance 
Model 

X 

CAM-I  Capacity 
Model 

X 

X 

X 

X 

X 

CUBES  Model 

X 

X 

X 

Cost  Containment 
Model 

X 

X 

Gantt  Idleness  Charts 

Supplemental  Rate 
Method 

X 

Theory  of  Constraints 
Capacity  Model 

X 

Normalized  Costing 
Approach 

X 

X 

ABC  and  Capacity  Cost 
Measurement 

X 

Integrated  TOC- 
ABC  Model 

X 

X 

X 

Table  4.2  NAWCAD  Objectives  Vs  Model  Characteristics 
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CAM-I  CAPACITY  MODEL 


A.         CAM-I  CAPACITY  MODEL  REVIEW 

The  CAM-I  capacity  model  is  primarily  a  strategic  communication  tool,  designed 
to  support  the  strategic  decision  process  by  helping  managers  understand  and  define  the 
many  states  of  capacity,  measure  these  states,  and  then  communicate  them  in  a  simple 
format  (McNair  and  Vangermeersch,  1996).  This  economic  model  is  a  tool  that  can  be 
used  to  improve  the  productivity  of  existing  capacity  and  significantly  influence  the 
capital  investment  decision  process.  Based  on  the  CAM-I  Capacity  Interest  Group 
publication  Capacity  Measurement  &  Improvement,  A  Managers  Guide  to  Evaluating 
and  Optimizing  Capacity  Productivity  (1996),  an  overview  of  the  model  is  presented  here 
to  provide  a  baseline  understanding  of  its  components  and  application  considerations. 

There  are  several  concepts  at  the  core  of  the  model  design  that  need  to  be  defined. 
The  model  uses  a  baseline  measure  of  maximum  capacity  termed  "rated  capacity".  The 
common  element  defining  use  of  capacity  is  time.  Rated  capacity  is  a  time  measure 
based  on  twenty-four  hours  a  day.  The  cost  of  this  capacity  is  1 00  percent  of  the  total 
cost  assignable  to  the  process.  The  model  provides  a  framework  from  which  to  translate 
capacity  utilization  from  standard  operational  units  of  time  into  financial  units  of  dollars 
(cost).  This  is  accomplished  through  the  use  of  a  basic  template  that  displays  capacity 
utilization  as  a  function  of  time  divided  into  three  distinct  categories:  (1)  idle  capacity,  (2) 
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nonproductive  capacity,  and  (3)  productive  capacity.  Financial  data  can  then  be  assigned 
to  each  of  the  categories  indicating  distribution  of  process  costs.  (Klammer,  1996) 

The  model  subdivides  total  rated  capacity  into  more  specific  elements  of  capacity. 
The  summary  capacity  model  shown  in  Figure  5.1  is  an  example  of  the  basic  template 
used  to  subdivide  total  or  rated  capacity  into  idle,  nonproductive,  and  productive 
capacity. 


o 
a, 
U 

<U 

<-> 

o2 

Idle 

Nonproductive 

Productive 

Figure  5.1  CAM-I  Summary  Capacity  Model 
The  summary  model  can  be  expressed  as: 

Rated  Capacity  =  Idle  +  Nonproductive  +  Productive  Capacity 

The  full  capacity  model,  shown  in  Figure  5.2,  further  divides  idle,  nonproductive,  and 
productive  capacity  into  specific  classes  providing  a  greater  level  of  detail  for  identifying 
more  specific  uses  of  rated  capacity. 
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Summary 

Industry  Specific 

Strategv  Specific 
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Idle 

Not  Marketable 

Excess  not  Usable 
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Management  Policy 

Marketable 

Idle  but  Usable 

Non- 
productive 

Maintenance  and 
Setups 

Scheduled 
Unscheduled 

Standby 

Variability 
Suppliers/Customers 

Waste 

Scrap  and  Rework 

Productive 

Process  Development 

Product  Development 

Good  Products 

Figure  5.2  CAM-I  Full  Capacity  Model 
The  elements  of  the  full  capacity  model  are  defined  below: 


Idle  capacity  -  Capacity  not  currently  scheduled  for  use. 

The  CAM-I  Model  breaks  idle  capacity  into  three  specific  classes: 


1. 


2. 


j. 


Idle  not  marketable:  no  market  exists  or  management  made  a  strategic 
decision  to  exit  the  market.  This  capacity  is  a  target  for  abandonment. 
Idle  off  limits:  capacity  unavailable  for  use  because  of  holidays, 
contract,  or  management  policies  or  strategies. 
Idle  marketable:  a  market  exists  but  capacity  is  idle. 


Nonproductive  capacity  -  Capacity  not  in  a  productive  state  or  not  in  one  of 
the  defined  idle  states.  Nonproductive  capacity  includes: 

1 .  Setups  and  maintenance:  scheduled  and  unscheduled  downtime 

2.  Standby:  nonproductive  because  of  variability  caused  by  suppliers, 
customers,  or  internal  operations. 

3.  Waste:  may  be  scrap,  rework,  and  yield  loss. 

Productive  capacity  -  Capacity  that  provides  value  to  the  customer. 
Productive  capacity  results  in  the  delivery  of  good  products  or  services.  It 
may  also  represent  the  use  of  capacity  for  process  or  product  development. 
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The  summary  capacity  model  (Figure  5.1)  is  incorporated  as  columns  one  and  two 
of  the  full  model  (Figure  5.2).  The  summary  capacity  model  provides  the  general  state  of 
capacity  in  a  form  that  is  useful  for  decision-makers.  This  information  could  be 
presented  for  an  entire  organization,  a  plant,  a  process,  a  production  center,  a  machine,  or 
an  individual  (Klammer,  1996).  The  full  capacity  model  provides  a  comprehensive 
analysis  of  various  states  of  capacity.  The  detailed  specification  of  the  types  of  capacity 
is  particularly  useful  for  business  and  operating  teams  focusing  on  using  capacity  more 
effectively.  (Klammer,  1 996)  Two  basic  templates  are  used  to  display  capacity  utilization 
in  units  of  time  and  cost.  A  basic  time  template  is  shown  in  Figure  5.3.  The  basic 
economic  template  is  constructed  by  adding  process  costs  to  the  raw  time  data  and  is 
shown  in  Figure  5.4.  This  information  is  particularly  useful  to  management  for 
evaluation  of  capacity  utilization  and  productivity.  The  model  provides  clear 
communication  of  operational  capacity  utilization  in  terms  useful  for  business  decision- 
making. (Klammer,  1996) 
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Figure  5.3  CAM-I  Basic  Time  Template 


Equipment  Set  A 


Idle 
$200K 

Not  marketable 

Off-limits 

Marketable 

Standby 

Non"                     Maintenance 
productive 

$300K                  Sell'Ps 

Productive 
$500K 

Process  Development 

Product  Development 

Good  Products 

Figure  5.4  CAM-I  Basic  Economic  Template 
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B.         CAM-I  CAPACITY  MODEL  IMPLEMENTATION  AND  ADAPTATION 
FOR  RDT&E  LABORATORIES 

Successful  implementation  of  the  CAM-I  Capacity  Model  in  a  research  and 
development  environment  requires  a  clear  plan  of  action,  including  modification  of  the 
model  to  meet  specific  RDT&E  requirements.  Accordingly,  adjustments  to  the  model 
have  been  incorporated  in  this  thesis  to  address  the  unique  characteristics  of  the 
NAWCAD  RDT&E  laboratories.  The  goal  of  these  adjustments  is  to  provide 
measurement  criteria  that  will  accommodate  the  diversity  and  complexity  of  the  RDT&E 
laboratory  environment,  and  provide  accurate  information  useful  in  the  decision-making 
process.  This  section  addresses  model  implementation  procedures  and  highlights  unique 
considerations  for  the  RDT&E  laboratory  setting. 

Initial  model  development  should  use  historical  data  from  the  targeted  area  of 
capacity  management.  This  focus  helps  develop  an  understanding  of  the  current  capacity. 
Applying  the  model  to  projections  has  the  potential  to  provide  the  most  value.  The  model 
could  provide  input  to  product  investment  decisions,  capacity  authorization  decisions, 
strategic  supplier  management  decisions,  and  strategic  customer  management  decisions. 
(Klammer,  1996) 

1.  Model  Implementation 

The  CAM-I  Capacity  Interest  Group  recommends  a  series  of  steps  that  are  helpful 

in  implementing  the  capacity  model,  as  follows: 
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a)  Organize  the  implementation  team. 

b)  Determine  management  objectives. 

c)  Select  a  model  presentation  template. 

d)  Review  element  definitions. 

e)  Select  the  measurement  period. 

f)  Identify  and  access  operational  data. 

g)  Identify  and  access  financial  data. 

h)  Summarize  to  level  of  required  presentation  model. 

i)  Monitor  for  results. 

NAWCAD  management  as  part  of  a  comprehensive  implementation  plan  should 
consider  each  of  these  steps.  Further  discussion  of  these  steps  highlights  the  unique 
adaptation  considerations  for  the  NAWCAD  RDT&E  laboratory  environment,  and 
provides  additional  information  about  characteristics  and  capabilities  of  the  CAM-I 
capacity  model. 

a)         Organize  the  implementation  team 

Successful    implementation    of   the    capacity    model    requires    senior 
management  and  operation  teams  to  reach  a  consensus  on  the  need  for  the  model 
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information    (Klammer,    1996).       Model    presentation   in   this   thesis    should    assist 
management  in  determining  the  level  of  need  and  use  for  the  CAM-I  model  information. 

b)  Determine  management  objectives 

Effective  use  of  the  model  requires  identification  and  communication  of 
management  objectives.  The  objectives  may  include  idle  capacity  resolution 
(downsizing),  increasing  capacity  flexibility,  and  identifying  causes  of  variability  and 
waste  in  laboratory  activities.  Each  of  these  objectives  influences  the  activities  identified 
by  the  model  and  the  presentation  template(s)  (Klammer,  1996). 

c)  Select  a  model  presentation  template 

Different  capacity  templates  support  different  business  objectives.  For 
example,  if  idle  capacity  identification  is  important  to  the  organization,  basic  time  and 
economic  templates  (Figures  5.3  and  5.4)  can  identify  and  communicate  the  different 
levels  of  idle  capacity,  the  functional  areas  responsible,  and  the  costs  associated  with  that 
capacity. 

d)  Review  element  definitions 

The  model  uses  the  language  and  definitions  already  in  use  in  the 
organization.  However,  if  the  model  is  applied  to  multiple  entities,  i.e.  laboratories,  and  a 
common  language  does  not  exist  among  these  entities,  a  common  set  of  terms  needs  to  be 
established. 
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e)         Select  the  measurement  period 

The  model  focuses  on  strategic  decision  processes.  Measures  that  use 
quarterly  and  annual  periods  are  typically  more  useful  than  daily  and  weekly  reporting 
periods.  In  fact,  frequent  updates  may  contain  distortions  that  would  be  harmful  if  used 
to  make  strategic  decisions.  More  frequent  measures  may  have  value  at  specific 
operational  levels.  (Klammer,  1996) 

f)  Identify  and  access  operational  data 

The  model  is  an  economic  mirror  of  existing  capacity  (Klammer,  1996). 
States  of  capacity  as  defined  by  operations  should  be  used  in  establishing  the  baseline  of 
data  to  be  input  into  the  model.  Operational  activities  that  influence  decisions  are 
therefore  highlighted  by  the  model  and  translated  into  an  economic  presentation  format. 

g)  Identify  and  access  financial  data 

The  model  allows  the  user  to  focus  on  a  subprocess  within  a  larger  process 
or  activities  within  a  process.  Organizations  with  ABC  systems  probably  already  have 
financial  data  for  these  processes  and  activities.  For  organizations  using  less,  accurate 
overhead  cost  assignments,  opportunities  for  improvement  exist.  Through  a  more 
detailed  reporting  of  the  different  states  of  capacity  the  capacity  model  can  help  a 
company  assign  overhead  to  the  most  appropriate  process  and  activity.  (Klammer,  1 996) 
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h)         Summarize  to  level  of  required  presentation  model 

Summary  data  are  important  to  ensure  that  each  level  can  understand 
capacity  effects  on  their  areas  of  responsibility.  Who  will  be  using  the  information 
determines  what  level  of  detail  is  required. 

i)  Monitor  for  results 

Monitoring  helps  determine  if  operational  and  financial  data  collection  and 
applications  are  accurately  taking  place.  A  plan  to  continually  improve  the  operational 
and  economic  data  should  be  established  as  an  ongoing  process. 

2.  Model  Adjustments 

The  theoretical  capacity  used  as  the  baseline  capacity  measure  by  the  CAM-I 

capacity  model  is  24  hours  a  day,  every  day.  The  laboratories  studied  are  government 
owned  and  operated.  Over  ninety  percent  of  the  scientists  and  technicians  who  operate 
the  laboratories  are  civilian  DOD  personnel  working  daytime  eight-hour  shifts.  (Collier, 
1998)  Under  the  present  conditions,  most  of  the  laboratories  studied  are  therefore 
occupied  and  operational  for  only  the  working  hours  described  above.  The  CAM-I 
baseline  capacity  measure  of  24  hours  a  day,  every  day  of  the  year,  is  adjusted  to  eight 
hours  per  day  /  five  days  per  week  to  represent  a  more  practical  baseline  of  maximum 
available  capacity  for  the  NAWCAD  RDT&E  laboratories.  With  this  adjustment,  total 
rated  capacity  is  defined  as  eight  hours  per  day  /  five  days  per  week,  excluding  holidays. 
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C.         CAM-I  CAPACITY  MODEL  SUMMARY 

The  CAM-I  Capacity  Model  is  designed  to  provide  as  much  or  as  little  detail  as 
management  requires.  The  adaptation  of  the  CAM-I  model  in  this  thesis  provides 
examples  of  the  NAWCAD  RDT&E  laboratory  data  presented  in  the  summary  capacity 
model  format  (Figure  5.1)  and  the  basic  time  and  economic  template  formats  (Figures  5.3 
and  5.4).  These  examples  demonstrate  the  model's  ability  to  effectively  capture  and 
communicate  capacity  utilization  information.  These  templates  represent  the  foundation 
from  which  other  templates  and  tools  can  be  generated,  providing  multiple  layers  of  detail 
for  capacity  utilization  analysis.  For  example,  a  template  isolating  idle  capacity  for  a 
single  process  or  piece  of  equipment  can  identify  specific  causes  and  costs  of  the  idle 
state  of  capacity  for  that  activity.  In  addition,  a  summary  model  of  overall  NAWCAD 
RDT&E  capacity  can  be  used  to  help  establish  and  monitor  strategic  infrastructure 
reduction  goals. 

The  data  requirements  for  detailed  presentation  of  all  of  the  additional  templates 
are  beyond  the  scope  of  the  thesis.  For  example,  accurate  utilization  data  at  the  Level  III 
organizational  level  can  provide  the  user  with  a  more  detailed  view  of  laboratory 
capacity.  Practical  constraints  of  time  and  resources  limited  this  research  to  the  Level  II 
organizational  level.  However,  NAWCAD  should  review  the  need  for  additional  levels 
of  capacity  reporting  and  implement  the  templates  and  tools  necessary  to  collect  and 
process  appropriate  decision-making  information.  Examples  of  additional  templates  are 
found  in  the  CAM-I  Capacity  Interest  Group  publication  Capacity  Measurement  & 
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Improvement,  A  Managers  Guide  to  Evaluation  and  Optimizing  Capacity  Productivity 
(1996). 

Capacity  utilization  presented  in  the  CAM-I  capacity  model  is  one  approach  to 
measuring  performance  for  RDT&E  laboratories.  Acquiring  accurate,  dependable  time 
and  cost  data  is  critical  to  proper  analysis.  A  database  of  accurate  cost  data  established  as 
part  of  the  financial  management  system  can  also  be  used  to  generate  additional  financial 
productivity  measures.  Chapter  VI  introduces  potential  financial  productivity  measures 
for  NAWCAD  RDT&E  facilities,  incorporating  existing  accounting  procedures  and 
applying  financial  ratio  analysis  to  determine  laboratory  productivity  in  terms  of  cost  and 
revenue  relationships. 
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VI.     RDT&E  FINANCIAL  PRODUCTIVITY  MEASURES 


A.         OVERVIEW  OF  RDT&E  FINANCIAL  PRODUCTIVITY  MEASURES 

In  an  effort  to  manage  Defense  activities  more  in  line  with  best  business  practices 
of  the  American  commercial  sector,  an  increased  emphasis  is  placed  on  improving 
productivity  through  cost  efficiency  and  recovery  of  full  costs  associated  with  activities 
and  products.  Recovery  of  costs  associated  with  RDT&E  laboratories  is  dependent  upon 
the  amount  of  revenue  generated  from  sponsor-funded  projects  and  contract  use  of 
laboratory  facilities  and  personnel.  Identifying  all  costs  and  revenues  associated  with 
individual  labs  is  essential  in  building  proper  performance  measures. 

The  NAWCAD  research  and  engineering  4.0  competency  laboratories  are 
financially  managed  as  Navy  Working  Capital  Fund  (NWCF)  accounts.  NWCF  accounts 
are  designed  such  that  the  revenue  generated  by  the  account  activity  is  sufficient  to  cover 
the  portion  of  costs  allocated  to  that  account.  Financial  productivity  for  NWCF  accounts 
can  therefore  be  a  measure  of  the  laboratory's  ability  to  recover  all  of  its  assigned  costs 
for  the  Fiscal  Year.  However,  the  present  system  does  not  assign  full  cost  to  the 
individual  laboratories.  Cost  items  such  as  depreciation,  general  and  administrative,  and 
facilities  overheads  are  not  included  as  costs  assigned  to  laboratory  NWCF  accounts 
(Runion,   1998).     Incorporating  full  costs  into  productivity  measures  would  provide 
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management  with  better  decision-making  information  about  the  financial  performance  of 
laboratory  facilities. 

The  recent  emphasis  on  improving  cost  efficiency  of  DOD  laboratories  and  test 
centers  has  resulted  in  the  initiation  of  the  development  and  implementation  of  a  Cost- 
Based  Management  Tool  (CBMT)  designed  to  capture,  display  and  archive  comparable 
cost  data  associated  with  the  operation  of  Defense  RDT&E  organizations.  This  Defense 
wide  program  has  been  directed  by  the  Deputy  Secretary  of  Defense  to  be  fully 
operational  by  the  end  of  Fiscal  Year  1998  (Memorandum  (i),  1997).  The  overall  goal  of 
the  Cost-Based  Management  Tool  is  to  provide  executive  level  visibility  of  the  full  costs 
associated  with  DOD  Laboratories  and  Test  &  Evaluation  Centers.  The  Cost-Based 
Management  Tool  is  not  designed  to  account  for  revenue. 


B.         RETURN  ON  OPERATIONS  INDEX  AND  OPERATING  MARGIN 

By  the  end  of  FY  1998,  NAWCAD  RDT&E  laboratories  will  implement  CBMT, 
and,  as  part  of  the  process,  will  begin  to  collect  detailed  cost  data  across  all  spectrums  of 
the  organization.  Until  implementation  of  CBMT  is  complete,  the  RDT&E  laboratory 
cost  data  are  limited  to  the  output  of  the  existing  financial  management  system.  Since  the 
present  system  for  RDT&E  operations  does  not  account  for  full  cost  recovery  of 
laboratory  activity,  the  full  cost  and  revenue  data  required  for  performance  measurements 
described  in  this  thesis  were  collected  from  sources  both  inside  and  outside  of  the 
existing  financial  management  system.   These  data  were  used  as  input  into  the  following 
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equation,  the  results  of  which  are  presented  in  Chapter  VII,  as  a  financial  performance 
indicator  designed  to  measure  individual  laboratory  productivity: 

Return  on  Operations  Index  (ROOI)     =    Total  Revenue  /  Total  Cost  of  Operations 

The  ratio  of  Total  Revenue/Total  Cost  of  Operations  is  an  indicator  of  laboratory 
productivity  from  a  NWCF  perspective.  It  identifies  what  percent  of  all  costs  associated 
with  a  specific  laboratory  are  offset  by  revenues  generated  by  the  laboratory.  Results  of 
less  than  1.0  represent  a  loss.  Additionally,  the  difference  between  revenue  and  full  cost 
represents  the  laboratory's  dollar  contribution  to  the  organization  as  a  whole  (operating 
margin),  highlighting  the  total  amount  of  dollar  surplus  or  loss  generated  by  laboratory 
operations.  For  example: 

Laboratory  4.X  FY1997$'s 

Total  Revenue  $800,000 

Total  Cost  of  Operations         $1,000,000 

Operating  Margin  -  $200,000 

ROOI    =     $800,000  /  $1,000,000  =  .8 

In  this  example,  only  80  percent  of  the  total  cost  of  operations  are  recovered  from 
laboratory  generated  revenue,  resulting  in  a  loss  of  $200,000  for  the  year. 

Laboratories  will  be  grouped  into  categories  of  similar  operating  characteristics  to 
enable  meaningful  analysis  and  comparison  of  performance.  The  ROOI  for  each 
Laboratory  will  be  rated  against  a  benchmark  ROOI  for  each  category.    The  benchmark 
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ROOI   represents  the  average  ROOI  for  the  laboratory  category.      This   approach 

incorporates  best  business  practices  similar  to  corporate  use  of  financial  ratio  analysis. 

Measuring  productivity  in  terms  of  cost  and  revenue  relationships,  and  comparing  the 

results  to  industry  standards,  is  a  common  practice  in  industry  (Maher,  1997).  Examples 

of  corporate  performance  ratios  include  return  on  investment,  return  on  total  assets,  and 

return  on  sales. 

RDT&E  facilities  managers  are  also  interested  in  utilization  issues  related  to 

space  allocation  and  capital  investment  decisions.     Productivity  per  square  foot  is  a 

common  measure  of  facility  utilization  used  in  corporate  merchandising  and  production 

industries.  As  long  as  appropriate  parameters  are  defined,  productivity  per  square  foot  is 

a  useful  comparison  and  decision  analysis  tool  for  facility  space  allocation  issues. 

Measuring  revenue,  cost,  and  operating  margin  per  square  foot  of  facility  space  allocated 

to  each  lab  allows  for  performance  comparison  from  a  space  utilization  perspective.  For 

example: 

Laboratory  4.X  FY  1997 

Total  Square  Footage  10,000  sq  ft 

Revenue     (per  sq  ft)  ($800,000  /  10,000  sq  ft)  =        $80.00  (per  sq  ft) 

Total  cost  (per  sq  ft)  ($1,000,000  / 10,000  sq  ft)  =      $100.00  (per  sq  ft) 

Operating  margin  (per  square  foot)  =     -  $20.00  (per  sq  ft) 

Facility  space  productivity  measures,  when  compared  across  all  labs  as  well  as  within 

specific  categories,  provide  managers  an  additional  financial  tool  for  space  allocation  and 

capital  investment  decision-making. 
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C.         PERFORMANCE  COMPARISON  CHART 

The  CAM-I  Capacity  model  and  the  productivity  measures  presented  thus  far  are 
tools  designed  to  provide  better  information  for  business  decision-making.  Effectively 
presenting  the  results  of  these  measures  is  critical  to  their  successful  implementation.  A 
Performance  Comparison  Chart,  combining  the  results  from  productivity  measures  and 
the  CAM-I  model,  is  shown  in  Figure  6.1 . 
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Figure  6.1  Sample  Performance  Comparison  Chart 
This  chart  presents  critical  capacity  utilization  and  productivity  information  in  a 
format  that  enables  analysis  across  two  dimensions  of  performance.  The  vertical  axis  (Y) 
displays  productivity  information,  and  the  horizontal  axis  displays  capacity  utilization 
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information.  When  plotted  in  reference  to  each  other,  the  relative  financial  strengths  and 
weaknesses  of  laboratory  capacity  utilization  and  productivity  are  highlighted.  Different 
regions  of  the  graph  identify  areas  of  financial  strength  or  weakness,  dependent  upon  the 
specific  data  and  scales  used.  In  this  example  (Figure  6.1),  ROOI  is  plotted  against  idle 
capacity.  The  upper  right  region  of  the  chart  indicates  strong  performance  in  both 
productivity  and  capacity  utilization  (Lab  A).  The  lower  left  region  indicates  weak 
performance  in  both  productivity  and  capacity  utilization  (Lab  C).  The  center  region 
indicates  mid-range  overall  performance  with  varying  combinations  of  productivity  and 
capacity  utilization  (Labs  B  and  D).  The  snapshot  of  laboratory  performance  provided  by 
this  chart  can  alert  management  to  areas  of  business  strength  and  weakness  from  a 
financial  management  perspective. 

A  comparison  of  performance  for  the  mid-range  Labs  B  and  D  in  Figure  6.1 
highlights  the  charts  ability  to  communicate  different  levels  of  performance  significant 
for  strategic  analysis  and  decision-making.  Lab  B  ROOI  is  less  than  1  with  20%  idle 
capacity,  while  Lab  D  ROOI  is  greater  than  1  with  35%  idle  capacity.  If  the  primary 
objective  for  management  is  to  maximize  use  of  capacity,  Lab  B  is  the  stronger 
performer,  and  if  financial  productivity  is  the  primary  objective,  Lab  D  is  the  stronger 
performer.  The  overall  performance  of  both  Labs  is  very  similar  if  the  two  performance 
parameters  are  given  equal  strategic  importance.  In  this  case,  the  chart  highlights 
differences  between  the  two  similar  Labs  and  alerts  management  to  areas  of  business 
strength  and  weakness  for  each  performance  measure. 
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The  specific  data  and  scale  used  for  each  axis  can  be  determined  based  on 
management  preferences.  Combinations  of  productivity  and  capacity  utilization 
measures  can  be  selected  from  any  of  the  measures  available.  Four  examples  of 
Performance  Comparison  Chart  options  are  presented  below: 
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Productivity  Measure 
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Management  can  also  indicate  on  the  chart  minimum  acceptable  levels  of  performance 
for  each  parameter.  For  instance,  NAWCAD  may  set  a  minimum  acceptable  ROOI  of  0.5 
for  its  laboratories.  The  region  on  the  chart  of  less  than  0.5  ROOI  can  be  highlighted, 
drawing  attention  to  any  result  that  is  out  of  the  acceptable  range.  Chapter  VII  provides 
specific  examples  and  analysis  of  the  Performance  Comparison  Chart  incorporating  the 
results  of  data  collected  for  this  thesis. 
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VII.     DATA 


A.         DATA  COLLECTION  METHODOLOGY 

This  section  addresses  data  collection  methods  used  for  the  NAWCAD  RDT&E 
research  and  engineering  (4.0)  activities  and  discusses  difficulties  in  measuring  capacity 
utilization  in  the  RDT&E  laboratory  setting. 

Research  and  development  is  not  a  production  type  of  activity.  Output  is  difficult 
to  define  and  is  not  constant.  This  makes  it  difficult  to  define  a  measure  of  output. 
However,  the  factors  that  influence  output  can  be  identified.  Equipment,  facilities,  and 
personnel  are  the  primary  resources  that  contribute  to  NAWCAD  RDT&E  laboratory 
output.  The  CAM-I  capacity  model  is  able  to  communicate  utilization  of  these  resources 
using  time  as  a  common  measure  of  activity,  while  applying  costs  to  the  different  types  of 
activities  to  provide  an  economic  analysis  of  utilization.  Determining  which  of  these 
factors  are  significant  indicators  of  capacity  utilization  for  laboratory  facilities  is  critical 
to  the  success  of  the  model  in  providing  useful  decision-making  information  to.  RDT&E 
managers. 

Research  and  development  activities  rely  heavily  on  the  expertise  and  creativity  of 
highly  skilled  personnel  to  produce  output.  Any  attempt  to  measure  the  capacity  of 
personnel  would  have  to  include  both  tangible  and  intangible  factors.  There  can  be  little 
doubt  that,  in  today's  knowledge-based  economy,  intangible  assets  (like  the  programming 
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know-how  of  Microsoft)  can  be  far  more  valuable  than  the  tangible,  "fixed"  assets  that 
dominate  conventional  balance  sheets  (Stewart,  1994).  Measuring  intangible  assets,  such 
as  intellectual  assets  of  research  scientists,  programmers,  and  technicians,  that  comprise 
the  organization's  available  resources  in  a  reliable,  comparable  way  is  very  difficult. 
Qualitative  methods  of  measuring  the  intellectual  capacity  of  personnel  in  the  RDT&E 
environment  should  be  recognized  as  a  viable  input  to  overall  capacity;  however  the 
focus  of  this  research,  in  accordance  with  NAWCAD  objectives,  will  concentrate  on 
measurement  of  tangible  factors  such  as  equipment  and  facility  use. 

The  recent  reports  calling  for  DOD  RDT&E  infrastructure  reductions  (GAO, 
1998),  along  with  the  implementation  of  the  Cost-Based  Measurement  Tool 
(Memorandum  (i),  1998),  have  focused  management  attention  on  the  need  for  accurate, 
reliable  data  on  the  use  and  costs  associated  with  RDT&E  infrastructure,  i.e.  facilities  and 
equipment.  Quantifiable  measures  must  be  used  to  provide  comparable  data  across  all 
laboratories.  The  CAM-I  capacity  model  is  designed  to  present  capacity  data  in  a  format 
that  facilitates  comparison  of  capacity  utilization  across  functions  and  activities,  while 
providing  a  tool  to  establish  internal  benchmarks  for  business  activities  where  industry 
standards  do  not  exist  (Klammer,  1996).  A  quantitative  approach  of  measuring  the  use  of 
laboratory  equipment  and  facilities  should  give  management  a  strong  baseline  of  capacity 
utilization  information  vital  to  capacity  and  resource  allocation  decision-making. 
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1.  Equipment  Utilization 

Unlike  major  production  facilities,  research  laboratories  utilize  thousands  of 
pieces  of  equipment  designed  for  specific  laboratory  functions.  The  laboratories  operate 
as  separate  entities,  functionally  aligned  to  support  overall  mission  requirements,  and 
house  the  equipment  necessary  to  support  their  particular  function.  The  types  of 
equipment  used  are  not  standard  from  one  laboratory  to  another.  Large  mechanical  test 
devices  such  as  horizontal  accelerators,  computers,  specialized  video  analysis  equipment, 
and  aircraft  cockpit  simulators  are  just  a  few  examples  of  the  variety  of  equipment  used 
in  the  laboratories. 

One  way  to  measure  equipment  use  in  a  standard,  comparable  format  is  by  using 
time  as  the  baseline  unit  of  measure.  Allocating  the  rated  capacity  (total  time  the 
equipment  is  available  for  use)  into  different  types  of  use  (productive,  nonproductive,  and 
idle)  describes  equipment  utilization  in  a  common  frame  of  reference.  Utilization  rates 
for  the  different  types  of  equipment  provide  an  indicator  of  overall  capacity  utilization  for 
the  laboratory.  When  compared  to  similar  laboratories  and  equipment,  the  relative  use  of 
equipment  for  each  laboratory  can  be  evaluated  as  one  indicator  of  overall  capacity 
utilization.  Accurate,  dependable  data  are  required  for  proper  analysis,  The  measurement 
of  equipment  utilization  in  terms  of  time  can  be  accomplished  through  a  manual  tracking 
system  that  records  equipment  activity  on  a  daily  basis.  Table  7.1  is  an  example  of  a  data 
collection  form  designed  to  record  daily  equipment  use.  (CMS,  1997) 
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Date: 
Equipment  #  : 

Setup 

Productive 

Maintenance 

Idle 

Standby 

Comments 

0800-0900 

0900-1000 

1000-1100 

1100-1200 

1200-1300 

1300-1400 

, 

1400-1500 

1500-1600 

Table  7. 1  Equipment  Use  Data  collection  form 
This  data  collection  form  is  presented  as  an  example  of  a  potential  method  for 
obtaining  equipment  utilization  information.  The  equipment  utilization  data  provided  for 
this  thesis  were  collected  from  distribution  of  a  survey  (Appendix  B)  and  did  not 
incorporate  use  of  the  time  sheet  shown  in  Table  7.1. 

2.  Time  Frame  of  Analysis 

The  time  frame  of  analysis  for  this  type  of  information  is  an  important 
consideration.  Research  and  development  laboratories  perform  many  different  tasks,  and 
equipment  use  is  driven  by  sponsor  funded  projects  and  overall  mission  objectives. 
RDT&E  projects  are  cyclical  and  have  no  standard  duration.  To  compare  utilization  rates 
on  a  daily  or  weekly  basis  would  be  misleading.  An  appropriate  time  frame  of  analysis 
that  captures  several  cycles  of  activity,  based  on  management  experience  and  knowledge 
of  variable  project  length  and  cycle  time,  is  three  years  worth  of  data  (Harris,  1998).  This 
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type  of  historical  equipment  utilization  data  is  not  available  under  the  present  system.  As 
a  result,  the  questionnaire  in  Appendix  B  was  used  to  gather  data  from  laboratory 
operators  and  managers  approximating  equipment  utilization  for  Fiscal  Year  1997.  The 
results  were  used  as  input  to  the  CAM-I  Capacity  model  and  are  presented  later  in  this 
chapter.  The  reports  generated  from  these  data  are  intended  to  demonstrate  model 
attributes  and  provide  management  with  insight  into  the  analysis  and  decision-making 
tools  available  for  measuring  capacity  utilization.  However,  given  the  data  collection 
process,  the  examples  in  this  thesis  may  not  represent  actual  utilization  rates  experienced 
in  Fiscal  Year  1997. 


B.         DATA  SOURCES  FOR  AIRCREW  SYSTEMS  (4.6)  LABORATORIES 

Input  for  the  models  and  measures  presented  in  this  section  were  gathered  from 
Aircrew  Systems  (4.6)  laboratory  data  collected  from  NAWCAD  accounting  records,  an 
independent  database  of  laboratory  information,  interviews,  observations,  and  a 
questionnaire  distributed  to  laboratory  managers.  The  two  areas  studied,  productivity  and 
capacity  utilization,  are  each  supported  by  different  types  of  research  data.  The 
productivity  measures  presented  in  this  chapter  use  historical  data  collected  from 
accounting  records  for  FY  1997  and  estimates  of  performance  for  FY  1998  and  1999. 
Additionally,  interviews  with  the  Aircrew  Systems  (4.6)  laboratories  general  manager 
(Harris,  1998)  and  Comptroller  (Runion,  1998)  provided  supplemental  information  about 
future  allocation  of  facility  production  overhead  costs  and  general  and  administrative 
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costs  not  presently  included  in  the  NWCF  FY  1997  laboratory  rate  calculations.  The 
specific  CAM-I  capacity  model  examples  of  capacity  utilization  presented  in  this  chapter 
use  data  collected  from  responses  to  the  questionnaire  shown  in  Appendix  B,  as  well  as 
cost  data  supplied  by  NAWCAD  and  the  independent  database.  Details  of  the  data 
collection  methods  used  are  presented  in  the  following  sections. 

1.  Capacity  Utilization 

Observation  of  laboratory  activity  and  interviews  with  Aircrew  Systems  (4.6) 
laboratory  managers  and  facility  supervisors  were  conducted,  providing  the  background 
information  necessary  to  develop  the  questionnaire  (Harris,  1998).  The  questionnaire 
asked  laboratory  managers  to  estimate  equipment  utilization  rates  experienced  in  FY 
1997.  It  also  prompted  respondents  to  identify  laboratories  by  types  of  RDT&E  activity, 
facilities,  and  equipment.  The  data  collected  from  the  questionnaire  are  based  on 
laboratory  managers'  experience  and  judgement  of  actual  FY  1997  activity.  Presentation 
of  this  data  in  the  CAM-I  Capacity  model  is  intended  to  provide  management  with  an 
example  of  the  model's  ability  to  communicate  utilization  information.  The  accuracy  of 
the  information  depends  on  the  ability  of  the  managers  to  recall  actual  .usage. 

The  responses  include  all  nine  of  the  Aircrew  Systems  (4.6)  Level  II  groups  of 
laboratories.  Each  of  the  Aircrew  Systems  (4.6)  managers  with  Level  II  responsibility 
estimated  equipment  utilization  rates  based  on  an  aggregate  of  laboratory  activity 
reported  by  the  appropriate  individual  Level  III  laboratories. 
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Each  level  represents  a  different  organizational  focus  of  analysis  for  capacity 
utilization  information.  Recent  studies  such  as  the  1997  Quadrennial  Defense  Review 
and  Joint  Vision  2 1  have  emphasized  that  DOD  RDT&E  activities  needed  to  standardize 
accounting  and  reporting  formats.  As  a  result,  NA WCAD  has  established  Level  II  as  its 
standard  baseline  for  reporting  RDT&E  laboratory  activity  (Collier,  1998).  Accordingly, 
the  present  NAWCAD  RDT&E  financial  management  system  is  designed  to  capture 
accounting  data  at  the  Level  II  organizational  level.  Adaptation  of  full  cost  accounting 
procedures  and  an  activity  based  accounting  system  in  the  future  may  facilitate  more 
detailed  accounting  and  reporting  of  laboratory  activity  at  lower  levels  of  the 
organization,  i.e.  Level  III  laboratories  (Collier,  1998).  The  CAM-I  model  examples  in 
this  thesis  represent  information  for  Level  II  laboratory  activity. 

2.  Productivity 

FY  1 997  historical  cost  and  revenue  data  were  obtained  from  sources  both  internal 

and  external  to  the  organization.  Eagan  McAllister  Associates,  Inc.  (EMA),  a  defense 
contractor,  has  been  working  closely  with  NAWCAD  facilities  management  to  help 
design  productivity  measures  for  the  Research  and  Engineering  (4.0)  laboratories 
(Collier,  1998).  A  database  was  established,  including  a  description  of  activities,  cost, 
revenue,  and  square  footage  of  space  allocated  to  each  laboratory.  The  EMA  database  is 
the  primary  source  of  direct  laboratory  costs  and  revenues  used  in  this  thesis.  Indirect 
costs  of  production  overhead,  utilities  and  general  and  administrative  were  not  included 
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in  the  EMA  database.  Capturing  the  full  cost  of  RDT&E  activity  requires  both  direct  and 
indirect  cost  identification.  As  a  result,  additional  accounting  data  were  collected  from 
the  NAWCAD  Comptroller's  office,  which  identified  Research  and  Engineering  (4.0) 
laboratory  production  overhead  and  general  and  administrative  costs.  Combining  the 
direct  costs  from  the  EMA  database  with  the  indirect  costs  applicable  to  Aircrew  Systems 
(4.6)  laboratories  allows  for  an  approximation  of  full  costs  for  Aircrew  Systems  (4.6) 
laboratory  activities  to  be  input  into  productivity  measures  presented  in  this  Chapter. 

Projected  FY  1998  and  1999  cost  and  revenue  figures  were  gathered  from  budget 
estimates  provided  by  Aircrew  Systems  (4.6)  laboratory  managers.  The  estimates 
represent  laboratory  activity  anticipated  from  future  DOD  sponsored  projects  and 
potential  non-DOD  contract  business.  Use  of  these  data  provides  a  multi-year  view  of 
actual  and  estimated  cost  and  revenue  distribution,  reducing  the  effect  of  short-term 
variability  in  laboratory  activity  on  overall  productivity.  These  cost  and  revenue  data 
were  used  as  input  into  the  productivity  measures  described  in  this  chapter  to  provide 
examples  of  the  types  of  productivity  tools  available  to  NAWCAD  managers. 


C.         CAM-I  CAPACITY  MODEL  PRESENTATION  OF  AIRCREW  SYSTEMS 
(4.6)  LABORATORY  EQUIPMENT  UTILIZATION  DATA 

Equipment  utilization  data  for  the  CAM-I  capacity  model  examples  presented  in 
this  section  were  collected  from  the  questionnaire  shown  in  Appendix  B.  A  summary  of 
responses  to  questions  about  equipment  utilization  is  presented  in  tabular  form  in  Table 

7.2. 
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Table  7.2  Aircrew  Systems  (4.6)  Laboratory  Equipment  Utilization  Data 
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Equipment  utilization  rates  are  represented  for  each  of  the  nine  Aircrew  Systems 
(4.6)  Level  II  laboratory  groups.  The  data  presented  in  table  7.2  were  used  to  construct 
CAM-I  capacity  model  templates  as  examples  of  Aircrew  Systems  (4.6)  capacity 
utilization.  Totals  for  all  of  Aircrew  Systems  (4.6)  laboratories  are  calculated  by 
averaging  the  results  of  the  nine  individual  Level  II  responses.  The  totals  represent 
overall  Level  I  Aircrew  Systems  (4.6)  equipment  capacity  utilization. 

Additional  data  describing  laboratory  characteristics  were  collected  from  the 
questionnaire  and  are  presented  in  Table  7.3.  Each  laboratory  is  categorized  by  facility 
type,  equipment  type,  and  function.  This  information  provides  the  framework  from 
which  comparisons  of  performance  can  be  made  among  laboratories  with  similar 
characteristics.  For  example,  laboratories  primarily  using  large  mechanical  equipment  in 
a  high  bay  mechanical  facility  may  exhibit  cost  and  activity  behavior  significantly 
different  than  laboratories  primarily  using  small  technical  equipment  in  a  raised  floor 
computer  and  electronics  facility.  Separate  benchmarks  of  performance  for  laboratories 
with  similar  characteristics  may  provide  better  decision-making  information  to  managers. 
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Table  7.3  Aircrew  Systems  (4.6)  Laboratory  Characteristics 
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Using  the  data  from  table  7.2,  examples  of  the  CAM-I  basic  time  template  shown 
in  Figure  7.1  were  constructed  for  the  following  Level  II  laboratories:  Advanced 
Crewstation  Technology  Lab,  Aircraft  Integration/Test  Labs,  and  Thermophysiology 
Research  Facility.  These  Level  II  laboratories  were  chosen  as  examples  to  demonstrate 
the  CAM-I  model's  ability  to  communicate  and  highlight  capacity  utilization  across  a 
diverse  spectrum  of  laboratory  types. 


Idle 

5% 

Marketable 

Productive 

80% 

Process  15% 
Development 

Product  5% 
Development 

Good  60% 
Products 

Idle 

16% 

Marketable 

Productive 

68% 

Process  11% 
Development 

Product  5% 
Development 

Good  53% 
Products 

Idle 

25% 

Productive 

65% 

Marketable 

Product     5% 
Development 

Good       60% 
Products 

Advanced  Crewstation 
Technology  Labs 


Aircraft  Integration/ 
Test  Labs 


Thermophysiology 
Research  facility 


Figure  7. 1  CAM-I  Basic  Time  Templates  for  Aircrew  Systems  Level  II  laboratories 
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Each  of  the  examples  in  Figure  7.1  represents  a  different  type  of  laboratory 
facility.  The  Advanced  Crewstation  Technology  Labs  primarily  operate  in  computer  and 
electronics  facilities,  the  Aircraft  Integration/Test  Labs  primarily  operate  in  general 
purpose  clean  lab  facilities,  and  the  Thermophysiology  Research  Facility  is  a  high  bay 
mechanical  facility.  The  different  states  of  capacity,  idle,  productive,  and  non-productive 
are  identified  as  a  percent  of  total  rated  capacity.  In  the  NAWCAD  adjusted  CAM-I 
model,  total  rated  capacity  represents  eight  hours  a  day,  5  days  a  week.  This  information 
can  also  be  presented  as  units  of  time,  i.e.  hours  and  minutes.  For  instance,  25  percent  of 
total  rated  capacity  represents  two  hours  of  time  out  of  the  eight  hours  available  per  day. 

Figure  7.1  illustrates  the  amount  of  time  that  capacity  is  in  an  idle  state  for  each  of 
the  laboratories.  The  Thermophysiology  Research  Facility  reported  that  one  quarter  of 
the  total  capacity  available  is  idle  and  marketable.  In  contrast,  the  Advanced  Crewstation 
Technology  Labs  reported  only  five-percent  idle  capacity.  Idle,  marketable  capacity 
represents  additional  business  opportunity.  It  also  highlights  areas  of  potentially 
underutilized  capacity.  Laboratories  with  high  levels  of  idle  capacity  are  tying  up 
resources  that  may  be  better  utilized  by  other  laboratories  constrained  by  capacity  limits. 
This  type  of  analysis  is  relevant  for  space  allocation  and  capital  investment  decision- 
making. 

The  examples  also  provide  insight  into  the  types  of  non-productive  and  productive 
capacity  used.  Identifying  non-productive  time  classified  as  standby  for  the  Aircraft 
Integration/Test   Labs   highlights   potential    inefficiencies    in    scheduling   or   material 
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handling  and  supply  systems.  A  comparison  of  productive  capacity  among  the  three 
examples  draws  attention  to  the  process  development  component  of  the  Advanced 
Crewstation  Technology  Labs  capacity  utilization.  Process  improvement  and 
development  may  be  critical  to  the  successful  operation  of  the  laboratory;  however,  it 
does  not  directly  produce  output  for  the  customer  and  is  not  included  in  the  utilization 
measure  defined  as  project  funded  laboratory  activity  or  "good  products".  Acquiring 
project  funding  and  producing  good  products  for  the  customer  are  essential  to  successful 
NAWCAD  laboratory  operations.  Maximizing  project  funded  laboratory  activity  can 
only  occur  if  idle  and  non-productive  capacity  utilization  is  minimized. 

The  differences  in  facility  type,  along  with  other  laboratory  characteristics,  may 
explain  some  of  the  variance  in  capacity  utilization  among  laboratories.  Table  7.3  lists 
each  Level  II  laboratory  by  operating  characteristics  of  facility  type,  equipment  type,  and 
functions  performed.  Categorizing  laboratories  by  their  operating  characteristics,  and 
comparing  performance  among  similar  types  of  laboratories,  may  provide  a  more  relevant 
baseline  of  comparison.  The  data  in  Table  7.3  are  used  later  in  this  chapter  to  establish 
specific  categories  of  laboratories  from  which  benchmarks  of  performance  are 
determined. 

The  capacity  information  conveyed  by  the  basic  time  template  (Figure  7.1)  relies 
on  information  about  capacity  utilization  from  the  operational  level.  This  presentation 
format  allows  for  identification  and  communication  of  capacity  use  among  Level  II 
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laboratory  groups.    The  next  step  is  to  translate  the  operational  time  data  into  financial 
information  useful  for  economic  decision-making  at  all  levels  of  the  organization. 

Applying  laboratory  costs  to  the  different  types  of  capacity  use  provides  the  basic 
economic  template  for  each  of  the  laboratory  groups.  Direct  and  indirect  costs  are 
combined  to  represent  a  full  cost  approach.  Direct  costs  include  those  for  labor, 
engineering  support,  spares,  maintenance,  contracts,  consumables,  utilities,  and  training. 
The  indirect  costs  added  are  production  overhead  and  general  and  administrative.  Table 
7.4  displays  Aircrew  Systems  (4.6)  Level  II  laboratory  costs  for  FY  1997. 


LEVEL  II  LABORATORY  TITLE 


Direct 

Costs 
(K$) 


Prod 
Ovhd 

(K$) 


G&A 

(K$) 


Total 
(Full  Cost) 

(K$) 


Advanced  Crewstation  Technology  Lab 
Aircraft  Integration/Test  Labs 
Aircrew  Altitude  Protection  &  Breathing 
Aircrew  Protection  and  Survival  Equipment 
Crashworthy  System  RDT&E  Facility 
Crew  Systems  Integration  Lab 
Crewstation  Transparency  and  Lighting  Lab 
Escape  System  RDT&E  Lab  Facility 
Thermophysiology  Research  Facility 


$1702 

$182 

$624 

$2,508 

$290 

$234 

$241 

$766 

$336 

$114 

$303 

$754 

$290 

$351 

$321 

$962 

$626 

$186 

$267 

$1,081 

$1332 

$38 

$401 

$1,771 

$282 

$72 

$499 

$854 

$433 

$122 

$441  , 

$996 

$109 

$177 

$71 

$358 

TOTAL  Aircrew  Systems 


$5400  $1477  $3100 


$10,050 


Table  7.4  FY  1997  Aircrew  Systems  (4.6)  Level  II  Laboratory  Costs 
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CAM-I  economic  templates,  shown  in  Figure  7.2,  communicate  the  results  of 
distributing  the  FY  1997  costs  to  idle,  non-productive,  and  productive  states  of  capacity. 


Idle  S  140  K 


Non-productive  S  419  K 


Productive  $  2,234  K 


Advanced  Crewstation 
Technology  Labs 


IdleS  98  K 


Non-productive  S  98  K 


Productive  $415  K 


Aircraft  Integration/ 
Test  Lab 


Idle  $  53  K 


Non-productive  S  21  K 


Productive  $  136  K 


Thermophysiology 
Research  Facility 


Figure  7.2  CAM-I  Economic  Templates  for  Aircrew  Systems  (4.6)  Level  II  Laboratories 
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Unlike  the  time  template,  where  equal  time  exists  for  each  capacity  set,  the  basic 
economic  template  for  each  increment  of  capacity  differs.  Another  characteristic  of  the 
economic  template  is  that  the  cost  for  time  in  idle  capacity  is  usually  less  than  an  equal 
amount  of  time  in  a  productive  state  (Klammer,  1996).  This  occurs  due  to  the  reduced 
personnel  and  maintenance  requirements  associated  with  idle  activity.  With  an  activity 
based  costing  system  in  place,  identifying  costs  for  each  state  of  capacity  should  be  a 
logical  extension  of  the  accounting  process.  Until  ABC  systems  are  implemented  for 
NAWC AD,  allocation  of  costs  to  the  different  states  of  capacity  should  be  determined  by 
facility  and  financial  managers.  The  examples  presented  in  Figure  7.2  distribute  costs  to 
the  different  states  of  capacity  in  direct  proportion  with  the  amount  of  time  assigned. 

The  capacity  information  conveyed  by  the  economic  templates  provides  managers 
with  financial  data  for  economic  analysis  and  decision-making.  A  summary  model  for  all 
of  the  Aircrew  Systems  (4.6)  laboratories  is  shown  in  Figure  7.3.  This  Level  I  summary 
model  is  an  example  of  the  CAM-I  model's  ability  to  communicate  capacity  utilization 
information  to  different  levels  of  the  organization.  Upper  level  management  may  be 
interested  in  reviewing  total  Level  I  capacity  figures  as  shown  in  Figure  7.3,  while  Level 
II  and  Level  III  templates  can  be  produced  to  provide  operational  level  managers  with 
more  detail. 
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Figure  7.3  Aircrew  Systems  (4.6)  Level  I  CAM-I  Summary  Model 
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D.         AIRCREW  SYSTEMS  (4.6)  LABORATORY  PRODUCTIVITY  DATA 
PRESENTATION 

The  laboratory  cost,  revenue,  and  square  footage  data  used  for  productivity 
measures  presented  in  this  section  were  collected  from  FY  1997  NAWCAD  accounting 
records  and  budget  projections  for  FY's  1998  and  1999,  as  well  as  selected  data  from  the 
Aircrew  Systems  (4.6)  database  developed  and  maintained  by  Egan  Mcallister  and 
Associates,  Inc.  Examples  of  Return  on  Operations  Index  (ROOI)  and  Operating  Margin 
calculations  are  presented  as  measures  of  productivity  for  the  Level  II  laboratories.  Cost, 
revenue,  and  operating  margin  per  square  foot  of  facility  space  allocated  are  also 
presented  providing  managers  additional  financial  tools  for  space  allocation  and  capital 
investment  decision-making. 

1.  Return  on  Operations  Index  (ROOI) 

The   ROOI   compares   laboratory  revenue   against  all   costs   associated   with 
laboratory  activity.  The  equation  used  to  calculate  ROOI  is: 
Return  on  Operations  Index  (ROOI)    =    Total  Revenue  /  Total  Cost  of  Operations 

Aircrew  Systems  (4.6)  Level  II  laboratory  cost  figures  for  FY  1997  and  projected  costs 
for  FY  1998  and  1999  are  shown  in  Table  7.5.  The  FY  1997  cost  figures  are  taken  from 
Table  7.4,  while  the  projected  costs  for  FY  1998  and  1999  are  budget  estimates  provided 
by  laboratory  managers.  Production  overhead  and  general  and  administrative  costs  for 
FY  1998  and  1999  were  allocated  based  on  budgeted  levels  of  activity. 
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97 

98 

99 

3-Yr 

Actual 

Projected 

Projected 

Avg  Annual 

Level  II  Laboratory  Title 

Cost  (K$) 

Cost  (K$) 

Cost  (K$) 

Cost  (K$) 

Advanced  Crewstation  Technology  Lab 
Aircraft  Integration/Test  Labs 
Aircrew  Altitude  Protection  &  Breathing 
Aircrew  Protection  and  Survival  Equipment 
Crashworthy  System  RDT&E  Facility 
Crew  Systems  Integration  Lab 
Crewstation  Transparency  and  Lighting  Lab 
Escape  System  RDT&E  Lab  Facility 
Thermophysiology  Research  Facility 


$2,508 

$2,990 

$2,695 

$2,731 

$766 

$999 

$981 

$915 

$754 

$821 

$827 

$800 

$962 

$1,327 

$1,744 

$1,344 

$1,081 

$2,673 

$2,339 

$2,031 

$1,771 

$1,232 

$1,269 

$1,424 

$854 

$1,122 

$1,227 

$1,068 

$996 

$1,884 

$1,892 

$1,591 

$358 

$1,333 

$1,465 

$1,052 

TOTAL  Aircrew  Systems 


$10,050         $14,381         $14,439  $12,956 


Table  7.5  Aircrew  Systems  (4.6)  Level  II  Laboratory  Costs 

Revenue  figures  for  FY  1997,  along  with  projections  for  FY  1998  and  FY  1999 
are  shown  in  Table  7.6.  A  comparison  of  the  full  cost  and  revenue  data  for  Aircrew 
Systems  (4.6)  Level  II  laboratories  for  FY  1997  is  represented  in  Figures  7.4  and  7.5. 
These  data  are  used  in  the  following  section  to  demonstrate  productivity  measures  such  as 
ROOI,  operating  margin,  and  operating  margin  per  square  foot. 
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Level  II  Laboratory  Title 


97 

Actual 

Revenue 

(K$) 

98 
Projected 
Revenue 

(K$) 

99 
Projected 
Revenue 

(K$) 

3-Yr 

Avg  Annual 
Revenue 

(K$) 

$7,270 

$7,596 

$3,597 

$6,154 

$2,071 

$1,246 

$1,220 

$1,512 

$1,855 

$1,783 

$1,829 

$1,822 

$2,148 

$3,513 

$2,756 

$2,806 

$1,827 

$3,261 

$2,854 

$2,647 

$3,194 

$2,549 

$2,605 

$2,783 

$710 

$525 

$522 

$586 

$2,526 

$2,787 

$2,757 

$2,690 

$1,220 

$2,016 

$2,016 

$1,751 

Advanced  Crewstation  Technology  Lab 
Aircraft  Integration/Test  Labs 
Aircrew  Altitude  Protection  &  Breathing 
Aircrew  Protection  and  Survival  Equipment 
Crashworthy  System  RDT&E  Facility 
Crew  Systems  Integration  Lab 
Crewstation  Transparency  and  Lighting  Lab 
Escape  System  RDT&E  Lab  Facility 
Thermophysiology  Research  Facility 


TOTAL  Aircrew  Systems 


$22,820         $25,275         $20,156 


$22,750 


Table  7.6  Aircrew  Systems  (4.6)  Level  II  Laboratory  Revenue 
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Figure  7.4  FY  1997  Aircrew  Systems  (4.6)  Laboratory  Costs 
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Figure  7.5  FY  1997  Aircrew  Systems  (4.6)  Laboratory  Revenues 
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The  ROOI  for  each  of  the  Level  II  laboratory  groups  was  calculated  using  the  cost 
and  revenue  data  from  Tables  7.5  and  7.6  and  is  presented  in  Table  7.7. 


97 

98 

99 

3-Yr 

Actual 

Projected 

Projected 

Average 

Level  II  Laboratory  Title 

ROOI 

ROOI 

ROOI 

ROOI 

Advanced  Crewstation  Technology  Lab 
Aircraft  Integration/Test  Labs 
Aircrew  Altitude  Protection  &  Breathing 
Aircrew  Protection  and  Survival  Equipment 
Crashworthy  System  RDT&E  Facility 
Crew  Systems  Integration  Lab 
Crewstation  Transparency  and  Lighting  Lab 
Escape  System  RDT&E  Lab  Facility 
Thermophysiology  Research  Facility 


2.90 

2.54 

1.33 

2.25 

2.70 

1.25 

1.24 

1.65 

2.46 

2.17 

2.21 

2.28 

2.23 

2.65 

1.58 

2.09 

1.69 

1.22 

1.22 

1.30 

1.80 

2.07 

2.05 

1.95 

0.83 

0.47 

0.43 

0.55 

2.54 

1.48 

1.46 

1.69 

3.41 

1.51 

1.38 

1.66 

TOTAL  Aircrew  Systems 


2.27 


1.76 


1.40 


1.76 


Table  7.7  Aircrew  Systems  (4.6)  Level  II  Laboratory  ROOI 

An  average  ROOI  for  the  three  years  of  data  (FY's  97,  98,  and  99)  is  presented  to 
demonstrate  the  affect  of  short-term  variability  in  laboratory  activity.  Any  single  year 
data  may  be  misleading  as  an  indicator  of  long-term  performance.  For  example,  the 
ROOI  calculated  for  the  Thermophysiology  Research  facility  for  FY  1997  is  3.41; 
however  the  projected  ROOI  for  the  next  two  years  is  lower.  The  three  year  average 
ROOI  of  1.66  is  more  representative  of  the  laboratories  overall  performance. 
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An  internal  benchmark  ROOI  provides  a  baseline  measure  of  laboratory 
performance.  Laboratories  of  similar  function  or  operating  characteristics  may  be 
grouped  together  to  establish  comparable  standards  of  performance.  As  discussed 
previously,  the  desired  time  frame  of  analysis  is  three  years  worth  of  data.  In  the  absence 
of  three  years  of  historical  data  for  the  Aircrew  Systems  (4.6)  laboratories,  a  combination 
of  1997  historic  data  and  1998-99  projected  figures  were  used  as  an  example  of  a  three- 
year  laboratory  activity  cycle.  The  Aircrew  Systems  (4.6)  laboratories  three-year  average 
ROOI  of  1.76  shown  in  Table  7.7  can  potentially  be  used  as  a  benchmark  for  the  group. 
Individual  laboratory  ROOI,  compared  with  the  benchmark  ROOI  for  that  group,  gives 
management  a  method  for  evaluating  laboratory  productivity.  The  example  shown  in 
Figure  7.6  compares  laboratories  against  an  Aircrew  Systems  (4.6)  benchmark  ROOI  of 
1.76. 
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Figure  7.6  Aircrew  Systems  (4.6)  Benchmark  ROOI  Comparison 

Categorizing  the  Level  II  laboratories  by  facility  type  and  determining  a  separate 
benchmark  ROOI  for  each  category  provides  more  specific  comparisons  of  performance 
as  depicted  in  Table  7.8. 
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FACILITY  TYPE 
ROOI  BENCHMARK 

High  Bay  Mechanical 

ROOI 

Craskworthy  Systems  RDT&E  Facility 

1.30 

Escape  Systems  RDT&E  Lab  Facility 

1.69 

Thermophysiology  Research  Facility 

1.66 

Category  Benchmark 
Computer  and  Electronics 

1.55 

Advanced  Crewstation  Technology  Lab 

2.25 

Crew  Systems  Integration  Lab 

1.95 

Crewstation  Transparency  and  Lighting  Lab 

0.55 

Category  Benchmark 
General  Purpose  Clean  Labs 

1.59 

Aircraft  Integration/Test  Labs 

1.65 

Aircrew  Altitude  Protection  &  Breathing  System  Facility 

2.28 

Aircrew  Protection  and  Survival  Equipment  RDT&E 

2.09 

Category  Benchmark 

2.01 

Table  7.8  Aircrew  Systems  (4.6)  Level  II  Laboratory  Facility  Type  Benchmark  ROOI 

A  closer  look  at  the  data  reveals  that  the  highest  three-year  average  ROOI  figure 
belongs  to  the  laboratories  categorized  as  general-purpose  clean  laboratories.  The 
average  ROOI  for  this  category  is  2.01.  In  contrast,  the  average  ROOI  for  the  computer 
and  electronics  laboratories  is  1.59.  This  is  one  example  of  the  type  of  categorization  that 
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is  available  for  management  review.    Management  preferences  and  further  research  can 
help  determine  the  most  useful  comparison  categories. 

2.  Operating  Margin 

The  operating  margin  for  each  laboratory  is  the  difference  between  revenue  and 
full  cost.  ROOI  calculations  discussed  in  the  previous  section  describe  laboratory  cost 
and  revenue  relationships  as  a  percentage,  whereas  operating  margin  represents  the  same 
relationship  as  a  total  dollar  amount  contributed  to  the  organization  as  a  whole.  Table  7.9 
lists  the  operating  margin  for  Aircrew  Systems  (4.6)  Level  II  laboratories  for  FY  1997 
and  projected  operating  margin  amounts  for  FY  1998  and  1999  using  the  cost  and 
revenue  data  from  Tables  7.5  and  7.6. 

There  are  differences  in  the  dollar  amount  of  operating  margin  provided  by  each 
of  the  Level  II  laboratories.  The  Advanced  Crewstation  Technology  Lab  contributed 
$4,761  K  in  FY  1997,  while  the  Crewstation  Transparency  and  Lighting  Lab  actually  lost 
money  and  was  a  financial  drain  on  the  organization  in  the  amount  of  $145K.  Once 
again,  a  three-year  look  at  the  data  provides  a  better  indication  of  trends  and  overall  long- 
term  performance.  In  this  case,  the  Advanced  Crewstation  Technology  Lab  remains  the 
largest  dollar  contributor  of  the  group  for  the  three-year  period;  yet  a  trend  of  decreasing 
operating  margins  is  projected  for  years  1998  and  1999.  Increasingly  negative  operating 
margins  are  also  projected  for  the  Crewstation  Transparency  and  Lighting  Lab.    This 
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operating  margin  analysis  identifies  financial  strengths  and  weaknesses  in  terms  of 
operational  cash  flow. 


Level  II 
Laboratory  Title 

Advanced  Crewstation  Technology  Lab 
Aircraft  Integration/Test  Labs 
Aircrew  Altitude  Protection  &  Breathing 
Aircrew  Protection  and  Survival  Equipment 
Crashworthy  System  RDT&E  Facility 
Crew  Systems  Integration  Lab 
Crewstation  Transparency  and  Lighting  Lab 
Escape  System  RDT&E  Lab  Facility 
Thermophysiology  Research  Facility 

TOTAL  Aircrew  Systems 


97 

98 

99 

3-Yr 

Actual 

Projected 

Projected 

Avg  Annual 

Operating 

Operating 

Operating 

Operating 

Margin  (K$) 

Margin  (K$) 

Margin  (K$) 

Margin  (K$) 

$4,761 

$4,605 

$901 

$3,423 

$1,305 

$247 

$239 

$597 

$1,101 

$962 

$1,002 

$1,022 

$1,186 

$2,186 

$1,012 

$1,462 

$746 

$588 

$515 

$616 

$1,423 

$1,317 

$1,336 

$1,359 

-$145 

-$597 

-$705 

-$483 

$1,530 

$903 

$865 

$1,099 

$862 

$683 

$551 

$699 

$12,771 


$10,895 


$5,717 


$9,794 


Table  7.9  Aircrew  Systems  (4.6)  Level  II  Laboratory  Operating  Margin 

3.  Productivity  per  Square  Feet 

Measuring  cost,  revenue,  and  operating  margin  per  square  foot  of  facility  space 
allocated  to  each  laboratory  allows  for  a  space  utilization  performance  comparison.  Table 
7.10  lists  the  total  number  of  square  feet  assigned  to  each  of  the  Aircrew  Systems  (4.6) 
Level  II  laboratories  and  provides  three-year  average  cost,  revenue,  and  operating  margin 
amounts  per  square  foot.  Comparing  the  operating  margin  (per  square  foot)  of  each  Level 
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II  laboratory  shown  in  Table  7.10  with  the  total  operating  margin  dollar  amounts  shown 
in  Table  7.9  illustrates  the  different  perspective  that  this  type  of  analysis  provides. 

Although  the  Crew  Systems  Integration  Lab  average  annual  operating  margin  is 
$1,359  K,  the  third  highest  total  for  Aircrew  Systems,  its  $990  operating  margin  (per 
square  foot)  was  the  highest  productivity  per  square  foot  of  all  nine  Level  II  laboratories. 
From  a  financial  perspective,  the  Crew  Systems  Integration  Lab  utilizes  its  space  more 
productively  than  the  other  Aircrew  Systems  Level  II  laboratories. 


Level  II  Laboratory  Title 


Square  Feet 
Assigned 

3  Yr  Avg 

Cost  /  per 

Square 

Foot 

3  Yr  Avg 

Revenue / 

per  Square 

Foot 

3  Yr  Avg 

Operating 

Margin  /  per 

Square  Foot 

6,623 

$412 

$929 

$517 

8,519 

$107 

$178 

$71 

4,143 

$193 

$440 

$247 

12,756 

$105 

$220 

$115 

6,768 

$300 

$391 

$91 

1,373 

$1,037 

$2,027 

$990 

2,616 

$408 

$224 

-$184 

4,425 

$359 

$608 

$249 

6,438 

$163 

$272 

$109 

Advanced  Crewstation  Technology  Lab 
Aircraft  Integration/Test  Labs 
Aircrew  Altitude  Protection  &  Breathing 
Aircrew  Protection  and  Survival  Equipment 
Crash  worthy  System  RDT&E  Facility 
Crew  Systems  Integration  Lab 
Crewstation  Transparency  and  Lighting  Lab 
Escape  System  RDT&E  Lab  Facility 
Thermophysiology  Research  Facility 


TOTAL  Aircrew  Systems 


53,661 


$241 


$424 


$183 


Table  7.10  Aircrew  Systems  (4.6)  Level  II  Laboratory  Productivity  (per  square  foot) 
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E.         AIRCREW  SYSTEMS  (4.6)  LABORATORY  PERFORMANCE 
COMPARISON  CHART 

Examples  of  capacity  utilization  and  productivity  measures  have  been 
demonstrated  separately  in  the  previous  sections.  The  performance  comparison  chart 
incorporates  the  results  of  capacity  utilization  and  productivity  into  a  format  that  enables 
analysis  across  both  dimensions  of  performance.  Using  the  information  from  Table  7.2 
and  Table  7.7,  a  performance  comparison  chart  is  presented  in  Figure  7.7. 

The  chart  displays  the  relative  strengths  and  weaknesses  of  the  three  previously 
selected  Level  II  laboratory  examples.  The  X-axis  represents  the  percent  of  rated 
capacity  that  has  been  identified  as  idle.  The  Y-axis  is  the  ROOI  calculated  for  each 
Level  II  laboratory.  The  lighter  shaded  area  in  the  upper  right  portion  of  the  chart 
indicates  strength  in  both  productivity  (ROOI)  and  capacity  utilization  (low  idle 
capacity).  The  dark  shaded  lower  left  portion  of  the  chart  indicates  potential  weakness 
across  both  dimensions  of  performance.  From  the  chart,  the  user  can  determine  the 
relative  performance  strengths  and  weaknesses  among  the  three  laboratories.  The 
Advanced  Crewstation  Technology  Lab  displays  strong  productivity  and  capacity 
utilization  relative  to  the  other  two  laboratories.  Each  of  the  other  two  laboratories  has 
similar  productivity  with  varying  levels  of  capacity  utilization.  Capacity  utilization  for 
the  Thermophysiology  Research  Facility  is  the  weakest  of  the  three,  yet  the  laboratory  is 
maintaining  an  ROOI  of  over  1.5. 
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Figure  7.7  Aircrew  Systems  (4.6)  Laboratory  Performance  Comparison  Chart 

ROOI  Vs  Idle  Capacity 
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The  chart  illustrates  that  the  three  laboratories  in  Figure  7.7  are  financially 
productive  given  their  reported  levels  of  capacity  utilization  but  opportunities  for 
improvement  in  both  dimensions  of  performance  are  identified.  For  example,  a  decrease 
in  idle  capacity  for  the  Thermophysiology  Research  Facility  should  result  in  more 
efficient  utilization  of  laboratory  resources.  An  associated  increase  in  laboratory 
productivity  may  result  in  a  higher  level  of  productivity  as  measured  by  ROOI. 
Improvement  in  both  dimensions  of  performance  will  move  the  laboratory  up  and  to  the 
right  on  the  performance  comparison  chart. 

A  comparison  of  all  Aircrew  Systems  (4.6)  Level  II  laboratories  is  presented  in 
Figure  7.8.  In  this  performance  comparison  chart  example,  ROOI  is  the  measure  of 
productivity  and  productive  capacity  as  a  percent  of  total  rated  capacity  is  the  measure  of 
capacity  utilization.  The  orientation  of  the  chart  is  from  lower  left  to  upper  right  along  a 
spectrum  of  weak  performance  to  strong  performance.  The  chart  allows  management  to 
identify  and  compare  all  nine  Aircrew  Systems  (4.6)  Level  II  laboratories  across  both 
dimensions  of  performance.  The  Crewstation  Transparency  and  Lighting  Lab  stands  out 
as  a  low  productivity  laboratory  relative  to  other  (4.6)  laboratories,  while  the  Aircraft 
Integration/Test  Labs  and  Thermophysiology  Research  Facility  display  low  relative 
levels  of  capacity  utilization. 
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Figure  7.8  Aircrew  Systems  (4.6)  Laboratory  Performance  Comparison  Chart 

ROOI  Vs  Productive  Capacity 
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Different  combinations  of  performance  measures  may  be  used  to  construct 
performance  comparison  charts  that  concentrate  on  specific  business  areas.  For  example, 
managers  faced  with  space  allocation  decisions  may  use  operating  margin  or  operating 
margin  per  square  foot  as  the  financial  productivity  performance  measure.  Figure  7.9 
plots  the  operating  margin  for  each  of  the  (4.6)  Level  II  laboratories  against  productive 
capacity.  Figure  7.10  represents  a  similar  comparison,  but  the  productivity  measure  of 
operating  margin  (per  square  foot)  is  used  instead  of  total  operating  margin. 

The  differences  are  important  to  note  and  can  affect  the  user  analysis  of  laboratory 
performance.  The  two  charts  show  similar  results  for  most  of  the  laboratories  with  one 
notable  exception.  The  relative  positions  of  the  Advanced  Crewstation  Technology  Lab 
and  the  Crew  Systems  Integration  Lab  are  interchanged.  Analysis  of  relative  strengths 
and  weaknesses  of  these  two  laboratories  is  different  depending  upon  which  measure  is 
more  relevant  to  the  decision-making  process.  The  chart  identifies  the  Advanced 
Crewstation  Technology  Lab  as  the  stronger  performer  if  total  dollar  amount  of  operating 
margin  is  the  preferred  measure.  The  Crew  Systems  Integration  Lab  appears  to  be  the 
stronger  performer  if  operating  margin  (per  square  foot)  is  the  preferred  measure. 
Choosing  which  measures  to  use  can  impact  the  analysis  of  relative  strengths  and 
weaknesses  as  presented  by  the  performance  comparison  chart. 
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Figure  7.9  Aircrew  Systems  (4.6)  Laboratory  Performance  Comparison  Chart 
Operating  Margin  Vs  Productive  Capacity 
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Figure  7.10  Aircrew  Systems  (4.6)  Laboratory  Performance  Comparison  Chart 
Operating  Margin  (per  square  foot)  Vs  Productive  Capacity 
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VIII.    ANALYSIS 


A.         QUALITY  OF  DATA 

The  data  for  this  thesis  were  collected  from  three  primary  sources,  NAWCAD 

Aircrew  Systems  (4.6)  laboratory  accounting  records  for  FY  1997,  budget  estimates  for 
FY  1998  and  1999,  and  responses  to  the  questionnaire  shown  in  Appendix  B.  Examples 
of  capacity  utilization  measures  using  the  CAM-I  capacity  model  incorporated  laboratory 
manager  estimates  of  FY  1 997  equipment  utilization.  The  accuracy  of  the  estimates  is 
dependent  on  the  ability  of  the  managers  to  recall  the  level  of  equipment  utilization. 
However,  their  input  into  the  CAM-I  capacity  model  serves  to  illustrate  the  model's 
ability  to  convey  capacity  utilization  information  to  the  user.  Additional  observation  and 
systematic  reporting  of  equipment  utilization  is  necessary  to  generate  CAM-I  model 
results  for  management  decision-making. 

Productivity  measures  of  ROOI,  operating  margin,  and  operating  margin  per 
square  foot  incorporated  historical  accounting  data  for  FY  1 997  and  budget  estimates  for 
FY  1998  and  1999.  Production  overhead  and  general  and  administrative  costs  included 
in  the  total  cost  figures  were  provided  as  an  aggregate  of  all  research  and  engineering  4.0 
competency  laboratories.  Distribution  of  these  indirect  costs  to  each  of  the  nine  Level  II 
laboratories  studied  was  necessary  to  represent  the  full  cost  of  each  activity.  The 
NAWCAD  financial  management  system  does  not  presently  distribute  these  costs  to  the 
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individual  Level  II  laboratories.     Future  allocation  of  indirect  costs  to  NAWCAD 
RDT&E  laboratories  may  differ  from  the  allocation  methods  used  in  this  thesis. 

Analysis  of  laboratory  performance,  as  indicated  by  the  CAM-I  capacity  model 
and  productivity  measures  developed  for  this  research,  should  be  performed  using  three 
years  worth  of  data  to  reduce  the  effects  of  short  term  variability  in  laboratory  activity  on 
overall  laboratory  performance  results  (Harris,  1998).  One  year  total  cost  and  revenue 
figures  used  in  Chapter  VII  as  examples  of  FY  1997  laboratory  productivity  are 
representative  of  actual  results  but  should  not  be  considered  a  comprehensive  evaluation 
of  long  term  performance.  Budget  estimates  for  FY's  1998  and  1999,  used  as  indicators 
of  potential  future  performance,  were  included  in  the  presentation  of  productivity 
measures  to  illustrate  a  multi-year  analysis  of  data. 


B.         ASSESSMENT  OF  RESEARCH  RESULTS 

The  CAM-I  capacity  model  and  productivity  measures  described  in  this  thesis 
address  NAWCAD  RDT&E  management  objectives  concerning  the  evaluation  of 
laboratory  performance.  Each  of  the  performance  measurement  techniques  discussed  is 
capable  of  measuring  laboratory  activity  and  reporting  it  in  terms  useful  for  financial 
analysis.  Specific  methods  are  defined,  establishing  two  separate  dimensions  of 
performance,  capacity  utilization,  and  productivity.  Each  dimension  provides 
management  with  unique  information  about  laboratory  performance.     Combining  the 
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results  of  the  two  dimensions  of  performance  on  a  single  performance  comparison  chart 
provides  an  overall  perspective  of  laboratory  financial  strengths  and  weaknesses. 

Comparison  of  results  among  laboratories  facilitates  informed  decision-making  in 
areas  of  capital  investment,  laboratory  facility  resource  allocation,  operational  efficiency, 
and  RDT&E  infrastructure  reduction.  The  application  of  these  measures  in  the 
NAWCAD  RDT&E  laboratory  environment  was  established  through  examples  and 
analysis  of  Aircrew  Systems  (4.6)  Level  II  laboratory  data.  The  same  methods  applied  to 
the  Level  II  laboratories  in  this  research  are  potentially  applicable  across  all  levels  and 
types  of  DOD  RDT&E  activities. 

1.  Capacity  Utilization 

Using  the  CAM-I  capacity  model,  capacity  utilization  information  can  be 
communicated  to  all  levels  of  the  organization.  Different  types  of  laboratory  capacity 
utilization  and  their  associated  costs  are  identified  using  the  reporting  formats  offered  by 
the  model.  Accurate,  comparable  data  are  necessary  for  proper  analysis.  Comparison 
and  analysis  of  laboratory  capacity  utilization  data  are  available  using  the  CAM-I  basic 
time  and  economic  template  formats.  With  the  planned  development  and  implementation 
of  activity  based  costing  systems,  collection  of  accurate,  reliable  cost  data  will  be 
enhanced. 

Application  of  the  CAM-I  capacity  model  to  the  NAWCAD  RDT&E  laboratory 
environment  is  not  limited  to  any  one  measure  of  capacity  utilization.     Equipment 
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utilization  measures  described  in  this  research  represent  one  indicator  of  capacity 
utilization.  Other  viable  methods  of  measuring  laboratory  capacity  utilization  may  be 
incorporated  into  the  model  format.  The  ability  to  adapt  to  different  operating 
environments  is  one  of  the  primary  reasons  the  CAM-I  capacity  model  is  a  potential 
management  tool  for  the  NAWCAD  RDT&E  laboratories. 

Additional  measures  of  capacity  such  as  utilization  of  personnel  may  be  useful  in 
providing  a  more  comprehensive  evaluation  of  utilization.  Further  research  is  required  to 
determine  the  level  of  significance  that  equipment  utilization  rates  have  on  overall 
capacity  utilization.  This  research  was  conducted  on  the  assumption  that  equipment 
utilization  is  a  viable  indicator  of  laboratory  capacity  utilization.  However,  the  accuracy 
of  equipment  utilization  data  collected  for  this  research  has  not  been  validated.  The 
questionnaire  used  to  collect  equipment  utilization  data  asked  laboratory  managers  for 
estimates  of  FY  1997  equipment  utilization  based  on  experience  and  judgement.  More 
systematic  equipment  utilization  data  collection  methods  should  be  administered  to 
provide  information  for  decision-making. 

Data  collection  and  analysis  of  laboratory  activity  was  conducted  at  the  Level  II 
organizational  level.  The  CAM-I  model  is  capable  of  presenting  utilization  information 
at  the  level  of  detail  desired  by  the  user.  Level  III  laboratory  data  may  be  required  for  in 
depth  NAWCAD  RDT&E  financial  activity  analysis  and  decision-making.  The 
NAWCAD  financial  management  system  in  place  does  not  presently  account  for  Level 
III   laboratory  activity  at  the  detail  required  for  proper  identification  of  individual 
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laboratory  full  costs.  The  Level  II  data  collected  in  this  thesis  were  therefore  not  able  to 
identify  individual  Level  III  laboratory  performance. 

2.  Productivity 

The  productivity  measures  of  ROOI,  operating  margin,  and  operating  margin  per 
square  foot  focus  on  the  relationship  of  revenues  and  costs  associated  with  each 
laboratory.  This  approach  applies  financial  accounting  principles  of  full  cost,  net  profit 
or  loss,  and  operating  profit  margin  to  the  NWCF  account  structure.  Output  for  RDT&E 
laboratories  is,  therefore,  defined  in  financial  terms  instead  of  units  of  physical  product. 
Using  a  monetary  measure  of  productivity  allows  for  comparison  of  performance  across  a 
diverse  range  of  activities.  These  measures  can  be  applied  to  all  RDT&E  laboratories 
despite  their  operational  and  functional  diversity. 

Categorization  of  laboratories  allows  for  comparisons  of  productivity  among 
laboratories  with  similar  operating  characteristics.  Different  types  of  laboratories  display 
operational  characteristics  which  may  be  useful  in  establishing  category  benchmarks  of 
performance.  Three  distinctive  categories  of  laboratory  facility  type  were  used  to 
compare  performance  against  specific  category  benchmarks  for  the  Aircrew  Systems 
(4.6)  Level  II  laboratories.  The  results  indicate  a  potential  relationship  between  facility 
type  and  productivity.  A  larger  sample  of  laboratory  data  is  required  to  further  explore 
this  relationship. 
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RDT&E  facility  space  allocation  and  infrastructure  reduction  are  issues  of 
particular  concern  for  NAWCAD  RDT&E  management.  Operating  margin  per  square 
foot  of  space  allocated  focuses  management  attention  on  the  financial  productivity  of  the 
space  allocated  to  each  laboratory.  Examples  of  Aircrew  Systems  (4.6)  Level  II 
laboratory  operating  margin  per  square  foot  illustrate  the  relative  productivity  of  facility 
space  assigned  to  each  of  the  laboratories.  Similar  comparisons  can  be  established  for 
other  sets  of  laboratories.  Management  decision-making  and  review  of  laboratories 
competing  for  limited  facility  space  should  be  enhanced  with  productivity  information 
per  square  foot. 


C.         ADDITIONAL  CONSIDERATIONS 
1.         Non-financial  measures 

All  of  the  measures  identified  in  this  research  are  quantitative  and  comparable 
with  a  focus  on  the  financial  performance  of  RDT&E  laboratories.  Qualitative  and  non- 
financial  measures  of  performance  should  be  considered  as  part  of  any  comprehensive 
performance  analysis.  Although  a  particular  laboratory  may  indicate  poor  relative 
performance  based  on  the  measures  discussed  in  this  research,  a  review  of  non-financial 
factors  may  address  strategic  issues  not  captured  in  financial  performance  analysis.  For 
example,  some  of  the  laboratory  facilities  owned  by  NAWCAD  provide  troubleshooting 
and  technical  support  critical  to  fleet  operations.  Reduction  of  laboratory  functions  or 
resources  based  strictly  on  poor  financial  performance  may  jeopardize  the  fleet's  ability 
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to  operate  necessary  equipment  and  could  essentially  cripple  mission  readiness  in  the 
affected  operational  area. 

Additionally,  some  of  the  laboratory  facilities  are  considered  national  assets, 
which  provide  unique  capabilities  not  available  to  DOD  from  other  sources.  The  Aircrew 
Systems  Ejection  Seat  Tower  is  an  example  of  this  type  of  facility.  It  is  essentially  the 
only  one  of  its  kind  in  the  United  States  and  is  necessary  for  critical  fleet  ejection  seat 
testing  and  troubleshooting.  Specific  criteria  should  be  developed  to  evaluate  if  a 
laboratory  is  a  critical  asset  as  described  above.  Laboratories  identified  as  critical 
NAWCAD  RDT&E  facilities  should  be  noted  in  any  analysis  of  performance.  It  is  still 
useful  to  apply  performance  measures  to  these  laboratories,  but  the  additional  non- 
financial  information  may  be  equally  important  in  strategic  decision-making. 

2.  Cost/Benefit 

The  costs  associated  with  implementation  of  a  NAWCAD  capacity  management 
system  based  on  the  CAM-I  capacity  model  described  in  this  thesis  have  not  been 
determined.  The  level  of  effort  dedicated  to  system  design  and  implementation  will  be  a 
determining  factor  of  cost.  NAWCAD  has  already  obligated  approximately  two  million 
dollars  to  contract  for  non-DOD  expertise  in  initiating  development  of  an  activity  based 
costing  system  and,  specifically,  to  establish  a  database  of  RDT&E  laboratory  financial 
and  operating  characteristics  (Collier,  1998).  Similar  efforts  may  be  required  to  establish 
the  CAM-I  capacity  model  as  a  useful  tool  for  NAWCAD  RDT&E  laboratories. 
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Productivity  measures  described  in  this  thesis  are  a  natural  extension  of  activity  based 
accounting  system  data  and  should  incur  minimal  cost  and  effort  for  successful 
implementation. 

The  primary  benefit  of  using  the  measures  and  tools  presented  in  this  research  is 
better  decision-making  information.  NAWCAD  management  is  limited  in  its  ability  to 
compare  competing  laboratories  when  faced  with  facility  space  allocation  and  capital 
investment  decisions  (Collier,  1998).  Accurate,  timely  collection  and  presentation  of 
capacity  utilization  and  productivity  information  can  enhance  the  decision-making 
process,  potentially  improving  long-term  financial  performance  of  NAWCAD  RDT&E 
laboratories.  The  potential  dollar  savings  expected  from  enabling  better  laboratory 
resource  allocation  decisions  have  not  been  determined.  However,  considering  that 
NAWCAD  research  and  engineering  4.0  competency  laboratories  requested  twenty-seven 
million  dollars  for  capital  investment  in  FY  1 997  and  received  only  ten  million  (Collier, 
1 998),  proper  distribution  of  limited  funds  is  dependent  on  the  type  of  accurate,  timely 
laboratory  performance  information  that  this  research  has  described. 
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IX.     CONCLUSIONS  AND  RECOMMENDATIONS 


A.         SUMMARY 

The  emphasis  on  efficiency  and  sound  business  practices  from  a  financial 

management  perspective  mandates  that  the  Navy  evaluate  and  incorporate  appropriate 
performance  measurement  tools  for  RDT&E  laboratories.  Laboratory  capacity  utilization 
and  productivity  are  primary  indicators  of  performance.  Put  into  terms  useful  for 
financial  analysis,  these  tools  should  improve  the  quality  of  decision-making  information 
available  to  the  resource  manager.  This  thesis  has  identified  capacity  utilization  and 
productivity  measures  applicable  to  the  NAWCAD  RDT&E  laboratory  organization.  The 
same  measures  can  be  applied  across  a  diverse  spectrum  of  RDT&E  activities,  providing 
DOD  with  a  mechanism  to  evaluate  performance  of  competing  DOD  RDT&E  laboratory 
resources. 

This  remainder  of  this  Chapter  answers  the  research  questions  developed  in  the 
first  Chapter,  and  indicates  potential  areas  that  require  future  research. 


B.         RESEARCH  QUESTIONS 

1.  How  do  we  measure  capacity  utilization  and  productivity  of  DON 

RDT&E  facilities  in  terms  useful  for  financial  and  resource  allocation  decision 

making?    Industry  models  and  techniques  provide  guidelines  for  proper  performance 
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measurement  of  RDT&E  activity.  Based  on  the  objectives  of  the  NAWCAD  RDT&E 
organization  and  attributes  of  models  studied,  this  thesis  identified  the  CAM-I  capacity 
model  as  a  method  of  measuring  and  reporting  capacity  utilization  for  RDT&E 
laboratories.  Revenue  and  full  cost  relationships  were  identified  through  return  on 
operations,  operating  margin,  and  operating  margin  per  square  foot  calculations  as 
financial  productivity  measures  of  laboratory  performance.  Plotting  the  two  dimensions 
of  performance,  capacity  utilization  and  productivity,  together  on  a  performance 
comparison  chart  provides  a  financial  management  tool  for  strategic  resource  allocation 
decision-making. 

2.  What  are  the  plausible  methods  of  measuring  capacity  utilization  and 

productivity  for  RDT&E  facilities?  Twelve  separate  capacity  utilization  models  shown 
in  Appendix  A  were  reviewed  as  potential  models  for  NAWCAD  RDT&E  laboratories. 
Laboratory  equipment  utilization  was  identified  as  an  indicator  of  capacity  utilization  for 
individual  laboratories.  Equipment  utilization  data  input  into  the  CAM-I  capacity  model 
illustrated  the  model's  ability  to  communicate  capacity  utilization  information  in  terms 
useful  for  financial  decision-making.  Productivity  measures  were  developed  by  using 
accounting  concepts  and  ratio  analysis  procedures  commonly  found  in  industry.  Revenue 
and  cost  relationships  shown  as  a  ratio  in  the  return  on  operations  index  and  as  a  dollar 
amount  in  the  operating  margin  calculations  are  identified  as  plausible  productivity 
measures  for  individual  laboratories.  These  measures  are  closely  related  to  the  Navy 
Working  Capital  Fund  account  structure  that  governs  NAWCAD  RDT&E  financial 
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operations.  Additionally,  the  model  and  measures  are  designed  to  integrate  with  activity- 
based  costing  (ABC)  systems  and  will  be  enhanced  by  the  additional  levels  of  accounting 
data  provided  by  ABC. 

3.  Can  existing  production  capacity  models  be  applied  to  non- 
production  environments  such  as  research  laboratories?  The  CAM-I  capacity  model 
was  identified  as  the  best  fit  for  the  unique  RDT&E  environment.  The  CAM-I  model  has 
been  used  in  both  production  and  service  industries.  Its  flexibility  in  capturing  and 
reporting  different  types  of  capacity  information  give  credibility  to  its  potential  use  in  the 
RDT&E  laboratory  environment. 

4.  Are  there  existing  Research  and  Development  benchmark 
performance  measures  in  industry?  No  standards  of  performance  were  found  with 
direct  application  to  the  NAWCAD  RDT&E  laboratories.  The  Boeing  Company  was 
identified  as  a  potential  source  for  benchmark  application,  but  lack  of  access  to  specific 
financial  and  operational  performance  information  limited  full  evaluation. 

5.  Can  similar  measures  be  applied  to  DON  RDT&E  laboratory 
facilities?  The  potential  exists  for  benchmark  application  of  performance  measures 
found  in  sources  external  to  NAWCAD.  Boeing,  NASA,  and  the  British  Defence 
Research  Agency  were  cited  as  organizations  exhibiting  RDT&E  laboratory  activity  and 
organizational  structure  similar  to  that  of  NAWCAD.  A  review  of  techniques  and 
methods  used  by  these  organizations  provided  multiple  approaches  to  performance 
measurement  of  laboratory  activity. 
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6.  Can  dissimilar  laboratories  be  classified  into  categories  useful  for 
financial  performance  comparisons?  NAWCAD  Aircrew  Systems  (4.6)  Level  II 
laboratories  were  classified  by  competency  and  facility  type  to  compare  performance 
against  category  averages  established  for  this  research.  Results  indicate  a  relationship 
between  facility  type  and  ROOI.  Classifying  laboratories  by  facility  type  may  provide 
management  with  better  decision-making  information  when  comparing  financial 
performance  of  functionally  dissimilar  laboratories. 

7.  Can  a  consensus  approach  to  measuring  capacity  utilization  and 
productivity  be  applicable  to  all  RDT&E  activities?  The  CAM-I  capacity  model  and 
productivity  measures  identified  in  this  research  are  applicable  to  all  RDT&E  activities 
operating  as  NWCF  accounts.  Additionally,  alternative  measures  of  capacity  utilization 
that  may  be  developed  in  the  future  can  be  incorporated  into  the  CAM-I  capacity  model 
to  accommodate  continued  discovery  in  the  study  of  RDT&E  capacity  utilization 
measurement. 


C.         SUGGESTIONS  FOR  FURTHER  RESEARCH 

Based  on  arguments  and  facts  presented  in  this  thesis,  the  following 
recommendations  are  offered  to  help  the  Naval  Air  Warfare  Center  Aircraft  Division  and 
the  Department  of  the  Navy  obtain  better  decision-making  information  and  performance 
from  its  limited  RDT&E  resources: 
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1 .  The  CAM-I  capacity  model  and  productivity  measures  used  in  this  thesis, 
applied  to  a  larger  sample  of  NAWCAD  Level  II  and  Level  III  laboratories,  will  provide 
NAWCAD  RDT&E  management  with  additional  levels  of  detail  and  analysis  from  which 
evaluation  of  research  results  can  be  significantly  enhanced.  The  results  from  this  thesis 
were  based  on  data  collected  from  the  nine  Aircrew  Systems  (4.6)  Level  II  laboratory 
groups.  A  larger  sample  of  Level  II  laboratories  and  data  collected  for  individual  Level 
III  laboratories  are  recommended  to  further  investigate  and  validate  the  potential  use  of 
the  measures  described  in  this  thesis. 

2.  Capacity  utilization  was  determined  in  this  research  by  an  estimate  of 
laboratory  equipment  utilization.  Further  observation  and  analysis  of  actual  equipment 
utilization  is  recommended  to  substantiate  the  findings  in  this  thesis.  The  time  sheet 
example  provided  in  Table  7. 1  could  be  used  as  a  data  collection  tool. 

3.  Further  research  is  recommended  to  identify  alternative  methods  of 
measuring  capacity  utilization.  Equipment  utilization  was  determined  in  this  research  to 
be  one  indicator  of  laboratory  capacity  utilization.  Other  viable  measures  may  exist.  For 
instance,  laboratory  personnel  were  considered  one  of  the  inputs  to  laboratory  capacity. 
A  measure  of  personnel  utilization  may  be  useful  in  determining  overall  laboratory 
capacity  utilization.  Obtaining  accurate  records  of  personnel  activity  within  the 
laboratories  may  be  difficult  or  impossible  under  the  current  accounting  system. 
Implementation  of  management  controls  designed  to  report  activity  of  key  laboratory 
personnel  might  be  required  to  provide  the  data  necessary  for  detailed  analysis. 


115 


4.  A  more  detailed  review  of  the  capacity  utilization  metric  and  laboratory 
performance  measures  developed  by  Boeing  may  provide  potential  benchmark  measures 
for  NAWCAD  and  other  DOD  laboratories.  In  the  absence  of  compatible  industry 
RDT&E  benchmark  measures,  NAWCAD  should  establish  internal  standards  of 
performance  for  laboratory  capacity  utilization  and  productivity.  Additional  data  are 
required  to  develop  appropriate  performance  standards. 

5.  Categorizing  laboratories  by  their  operating  characteristics,  and  comparing 
performance  among  similar  types  of  laboratories,  may  provide  a  more  relevant  baseline  of 
comparison.  For  example,  laboratories  primarily  using  large  mechanical  equipment  in  a 
high  bay  mechanical  facility  may  exhibit  cost  and  activity  behavior  significantly  different 
than  laboratories  primarily  using  small  technical  equipment  in  a  raised  floor  computer 
and  electronics  facility.  Separate  benchmarks  of  performance  could  be  established  for 
laboratories  with  similar  characteristics.  Three  distinctive  categories  of  laboratory 
facility  type  were  used  in  this  thesis  to  compare  performance  against  specific  category 
benchmarks  for  the  NAWCAD  Aircrew  Systems  (4.6)  Level  II  laboratories.  The  results 
indicate  a  potential  relationship  between  facility  type  and  productivity.  A  larger  sample 
of  laboratory  data  is  required  to  further  explore  this  relationship  and  to  determine  if  other 
performance  relationships  exist. 

6.  Integration  with  activity  based  costing  systems  will  provide  the  level  of 
detail  necessary  to  accurately  measure  individual  laboratory  performance  at  the  Level  III 
organizational  level.     Coordination  between  the  NAWCAD  ABC  initiatives  and  the 
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laboratory  performance  measurement  research  should  enhance  the  implementation  of 
future  financial  management  control  systems. 

7.  Further  research  is  recommended  to  investigate  and  develop  non-financial 
performance  measures  to  complement  the  financial  performance  measures  described  in 
this  thesis.  A  comprehensive  performance  evaluation  of  RDT&E  laboratory  activity 
should  not  be  limited  to  capacity  utilization  and  productivity  measures.  The  intellectual 
capacity  of  researchers  and  scientists,  value  chain  analysis,  and  identification  of  core 
competencies  are  a  few  examples  of  potential  non-financial  measures. 

8.  Identify  core  activities  vital  to  mission  and  fleet  support.  Some 
laboratories  may  provide  unique  functions  not  available  from  other  sources.  Others 
perform  functions  required  for  direct  fleet  support  that  are  considered  indispensable.  The 
capabilities  of  these  laboratories  cannot  be  reduced  or  eliminated  without  dramatically 
affecting  overall  mission  readiness  of  the  fleet.  Specific  criteria  for  evaluating 
laboratories  as  vital  to  mission  readiness  should  be  established.  An  official  list  of  these 
core  RDT&E  laboratories  could  provide  an  additional  layer  of  analysis  when  evaluating 
performance  of  individual  laboratories. 

9.  Coordinate  with  other  branches  of  the  Armed  Services  and  federal 
agencies  to  design  and  implement  a  consensus  approach  to  measuring  RDT&E  laboratory 
financial  performance.  The  results  of  this  thesis  and  other  relevant  reviews  may  be  used 
to  initiate  a  joint  research  effort  to  provide  DOD  with  standard  metrics  for  determining 
individual  laboratory  performance.     The  CBMT  is  a  service  wide  tool  developed  to 
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provide  executive  level  visibility  of  full  costs  for  DOD  RDT&E  activities.  An  extension 
of  the  CBMT  model  to  the  individual  laboratory  level  may  serve  as  a  catalyst  for 
development  of  service  wide  measures  of  individual  laboratory  capacity  utilization  and 
productivity. 
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APPENDIX  A 


COMPARISON  OF  CAPACITY  COST  MEASUREMENT  MODELS 
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APPENDIX  B 
AIRCREW  SYSTEMS  (4.6)  LEVEL  II  LABOARATORY  QUESTIONNAIRE 

Questionnaire 

Please  take  a  few  minutes  to  complete  the  enclosed  questionnaire  and  return  to  Don 
Harris  by  Friday,  20  March. 


•     Responses  should  be  based  on  ESTIMATES  of  actual  usage  rates  experienced  in 
FY97. 


•     Each  Lab  should  be  considered  as  one  entity. 

Exception:  (If  the  functions,  equipment,  and  physical  space  of  the  individual  lab  components  are 
significantly  different  from  one  another,  please  complete  a  separate  questionnaire  for  each  entity.) 


Lab  Title 

Name  of  Respondent 

Phone  # 

E-mail 
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LAB  DESCRIPTION 

1.    Lab  Facility  Physical  Space  Description 

Select  from  the  following  list  the  option  that  best  describes  the  physical  space  that  the  lab  occupies: 

High  Bay  Mechanical  (large  equipment  and  storage  labs) 

Computer  and  Electronics  (raised  floor  labs) 

General  Purpose  Clean  Lab 

Other  (describe) 


2.    Lab  Function  Type 

Consider  the  total  volume  of  work  performed  by  the  lab  in  FY97  and  estimate  the  amount  (%)  of  the  total 
that  was  spent  performing  the  following  functions: 

(Enter  0  if  the  listed  function  does  not  apply  to  the  lab) 

%  Production  (Lab  tasks  consist  of  repetitive,  defined  tasks) 

%  Research  and  Development  (R&D),  Research  Development  Test  and  Evaluation 

(RDT&E)  (Lab  tasks  are  not  well  defined,  nor  repetitive) 

%  Certification  (Lab  is  required  to  validate  aircraft  system  prior  to  flight,  flight 

clearance,  or  fleet  use) 

%  In-Service  (Lab  is  used  primarily  to  provide  direct  support  to  fleet  to  troubleshoot 

&  correct  problems) 

%  SSA  (Lab  is  a  software  support  activity) 

%  Other  (Describe) 


100    %  Total  Work  performed  by  the  lab 


3.  Lab  Equipment  Type 

Choose  the  option  that  best  describes  the  type  of  equipment  used  in  the  lab: 

Large  mechanical  test  device  (single  or  multiple) 


Technical  work  bench  areas  (multiple  work  areas  designed  for  tasks  utilizing  specific  equipment 
or  technology)  i.e.  electronic  test  benches,  simulation  and  modeling  computers,  video  analysis 
equipment. 

Non-Technical  work  bench  areas  (multiple  variable  use  work  areas  designed  for  a  variety  of  tasks 
not  restricted  to  specific  equipment  or  technology)  i.e.  open  lab  benches  for  material 
handling,  parts  and  inventory. 

Other  (Describe) 
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EQUIPMENT  UTILIZATION 

The  following  questions  ask  for  responses  about  utilization  of  the  Lab  equipment  identified  in  the  previous 
section.  The  questions  ask  you  to  identify  how  the  equipment  is  used  during  normal  working  hours  -  5 
davs/week,  one  8-hr  shift  -  not  including  holidays  or  other  legitimate  non-work  days. 
Note:  If  the  equipment  is  used  in  excess  of  normal  working  hours  on  a  Regular  Basis,  read  Note  1  at  the 
end  of  the  questionnaire. 

Equipment  use  is  divided  into  three  categories;  Productive,  Non-productive,  and  Idle 

Please  provide  the  appropriate  percentage  of  utilization  for  each  category.  The  sum  of  the  three  categories 
should  account  for  100%  of  the  equipment  usage  during  normal  working  hours. 

1.     Productive 

What  percentage  of  time  during  normal  working  hours  is  the  Lab  Equipment  used  for  the  following 
activities? 

A.  Producing  output  directly  tied  to  sponsor  funded  projects  (revenue  generating  projects) 

B.  Producing  output  not  tied  directly  to  sponsor  funded  projects  (work  initiated  to  gain  improved 

product  quality  or  new  product  development  but  not  linked  to  revenue) 
C.  Process  development  (work  not  associated  with  a  specific  product  or  revenue,  but  initiated  to 

improve  Lab  processes,  technology,  and  capability) 

%  Sub-total 


2.     Non-productive 

What  percentage  of  time  during  normal  working  hours  is  the  Lab  Equipment  used  for  the  following 

activities? 

A.  Required  preparation,  set  up  and  tear  down  NOT  funded  by  project  sponsors. 

B.  Scheduled  and  Unscheduled  Maintenance 

C.  In  Standby  mode  (The  equipment  has  been  set  up  and  prepared  for  scheduled  tasking  but  is 

not  presently  in  use  -  delay  time  between  set  up  and  actual  use) 

%  Sub-total 


3.     Idle 

What  percentage  of  time  during  normal  working  hours  is  the  Lab  equipment  NOT  being  used  for.  the 

following  reasons? 

A.  Restricted  use  of  Lab  equipment  due  to  Contractual  requirements,  Legal  requirements,  or 

Management  Policy  (Strategic  decision  to  protect  excess  capacity  for  mission  critical  contingency 

requirements) 

B.  Lack  of  Demand    *  Idle  but  Usable  (Available  for  additional  business,  market  opportunities) 

C.  Non-Marketable  Product/Output    *  Idle  but  NOT  Usable  in  its  present  condition  (No  market 

exists  for  Lab  equipment  capabilities;  equipment  may  be  outdated  or  requires  additional 

investment  or  upgrade) 

%  Sub-total 


Keep  in  mind  that: 

•  The  sum  of  percentages  given  in  questions  1,  2,  and  3  must  equal  100%. 

•  The  sum  of  percentages  for  parts  A,  B,  and  C  of  each  category  must  equal  the  section  sub-total. 
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Notel 

If  the  Lab  equipment  is  utilized  in  excess  of  normal  working  hours  on  a  REGULAR  BASIS,  limit  your 
responses  to  include  only  normal  working  hour  utilization  estimates.  In  the  space  provided  below, 
estimate  the  average  number  of  additional  hours  of  use  per  day  and  categorize  the  additional  use  of  the 
equipment  as  either  Productive  or  Non-productive. 

Average  Number  of  Additional  Hours  per  day: 

Productive  

Non-productive     
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